#markdown

• vd (visidata): Terminal spreadsheet viewer & editor for CSV, Excel, JSON, SQL, Parquet, etc.
• qsv: Fast CSV command line toolkit for slicing, filtering, aggregating, and analyzing CSV files.
• rga (ripgrep-all): ripgrep that searches PDFs, Office docs, EPUBs, zip files.
• pdfcpu: PDF processor for splitting, merging, optimizing, and manipulating PDF files.
• gum: Stylish CLI tool for creating interactive prompts, confirmations, and more.

function HorizontalRule()
  return pandoc.RawBlock('openxml', '<w:p><w:r><w:br w:type="page"/></w:r></w:p>')
end

• Document, lint, type-check, add test cases (or other similar tasks) for all folders in a monorepo.
• Research and create a report for each topic in */RESEARCH.md.
• Synthesize learnings from each conversation in transripts/*.md.

• Use AGENTS.md as guardrails, not a manual. Document what it gets wrong.
• Use self-documenting tools/APIs rather than documenting.
• Docs: Explain why and when to read each doc.
• Never say "Never." Explain when to which which alternative.
• Prefer CLIs for stateless tools, MCPs for stateful, authenticated, or complex (e.g. Playwright).

• Break up uncommitted changes into small commits
• Rewrite branch history for readability
• Use gh CLI to fetch line-wise comments from a PR and make requested changes (e.g. renaming, refactoring, adding types, etc.)

• Add custom prompts in ~/.codex/prompts/xyz.md and launch as /prompts:xyz. Optional: description: and argument-hint: in YAML front-matter. For example, create prompts to refactor, rewrite in a developer's style, document AGENTS.md, identify re-usable code, etc.
• AGENTS.override.md overrides parent directory AGENTS.md. AGENTS.md appends to parent AGENTS.md. Fallback names are allowed.
• codex exec supports streaming JSON
• codex exec accepts a CODEX_API_KEY= environment variable. codex uses an OPENAI_API_KEY.
• You can configure which environment variables are passed to the shell
• Codex reads 32KB from AGENTS.md by default

• codex --add-dir $DIR lets you write into $DIR
• codex --full-auto is the equivalent of codex --sandbox workspace-write --ask-for-approval on-request

• It's use-case / domain transfer: showing what it's useful for in real life - opening ideas, suggesting workflows.
• It's style transfer: showing how to use it.

const branch = await $`git branch --show-current`;
await $`dep deploy --branch=${branch}`;

• ChatGPT suggested increasing observability based on code reviews. I'm including it in my weekly codecast.
• ChatGPT suggested defining closures inmeetings. I'mn now discussing objectives at meeting starts and effectiveness at the end.

• It can run a terminal in its own new window for over a year (via Ctrl+P > Terminal: Move Terminal into New Window). Now, Ctrl + Alt + Shift + ` does this directly.
• Terminal Intellisense shows completion suggestions in the UI. Very helpful. Ctrl+Space triggers the menu completion.

• VS Code plugin with remote container execution
• Drag & drop image support PR Docs
• Queued (editable) messages PR
• Web search via --search PR
• Esc-Esc to edit previous messages Docs

• Summarize takeaways on top of each section
• Use atomic notes: one file per idea. Link liberally
• YAML front-matter you can query, e.g. tags, project, status, ...
• Use GFM admonitions, e.g. > [!NOTE]
• Store images in a predictable way, e.g. ![Alt text](./img/2025-08-21-screenshot.webp) -- ALWAYS with alt text
• Use diff fences for edits / doc changes
• Task lists with inline dates, e.g. - [ ] 2025-08-21 Draft a letter

1. User enters their scenario in textbox (e.g. "Synthetic data to demonstrate credit card fraud using transaction data. Mid-sized, multiple tables, complex. Excel file")
2. Click "Submit". This should send the scenario to an LLM asking it to
   • Think carefully and identify a schema, stating assumptions (e.g. number of rows to generate, table/column inter-relationships, output file format, etc.)
   • List typical real-world behaviors that could be incorporated into the data and will lead to positive results when testing these hypotheses (e.g. transactions over $10,000 are more likely to be declined, overseas transactions in low-income countries have high failure rates, etc.)
   • Write Python code that generates a random dataset in line with the above, writes the output to a bytestream and returns the filename and bytestream
3. Stream the response. Render it as markdown. Use highlightjs to syntax-highlight the code block using a dark theme.
4. Run the code in the browser using Pyodide, generate the output, and allow the user to download it
5. Allow the user to enter changes in their scenario. Have the LLM make changes to the code by sending the change and the original response. Repeat from step 3.

• Commands are used for:
  • Requirements: Research sub-agent, task breakdown into todos.md, creating specs.md from todos.md
  • Progress tracking: session logging, effort tracking, updating status, planning next steps
  • Project setup: initializing, adding deps, scaffolding features
  • Development: code review, debug error (five whys), explain code, refactor code
  • Optimization: optimize build, DB, caching
  • Testing: TDD, generate test cases, set up unit/integration/E2E testing, analyze coverage
  • Security: security audits, dependency vulnerability scans
  • Integration: sync tasks between GitHub and Linear (two-way issue synchronization, PR linking)
  • Deployment: prepare releases, hotfix deploys, rollbacks, containerization, CI pipeline setup
• Patterns of usage
  • Sub-agents
  • Command handoffs, i.e. one command invoking another
  • Shared among a team in a repo, enforcing standards & sharing best practices
  • Integration with specific tools / APIs (e.g. Linear)

• Delete-Driven Development. Keep deleting code until it fails, then rebuild minimally.
• No-Editor Coding. Pure prompting to build, test, and deploy the app.
• Audience-Sourced Prompt ETF. Collect micro-prompts from audience. Run in parallel. Evaluate against tests. Create index of top prompts. Compare with expert.

• Skip biomejs for now. It uses tabs (not spaces) and does not respect .gitignore by default. Handling these is too much work.

• Provide a stable, complete spec.
  • Late UI tweaks, new API params, renamed fields, extra packaging rules, “Rename per‑image download”, “standardise baseUrl vs baseURL”, “add GA‑4 exam module”. → churn & rewrites.
  • Ask the user for a final UI/API/mock‑up + edge‑case examples before the first commit.
  • Lock naming conventions, UI layout and feature checklist early; track future changes explicitly
• Include concrete examples.
  • Lack of sample images, Markdown snippets, question formats caused guesswork.
  • Supply mini‑fixtures: sample prompts, expected outputs, env‑var names, commit‑message template
• Environment should be reproducible.
  • E.g. vitest not installed, .dev.vars absent, sub‑modules not cloned, network blocks.
  • Ship a one‑step bootstrap script / README with npm install, env‑var templates, and submodule notes
• Automate tests.
  • First answer compiles but fails prettier/ruff/unit tests; later iterations fix style or red lines.
  • Codex should auto‑run lint && test (plus static‑analysis / self‑critique) before every response
• Auto-run post-mortems.
  • Codex recommending its own static checks shows value.
  • Automate that as a pre‑commit step.

• claude --debug shows what Claude Code is doing behind a scenes -- and is a good way to understand hidden / undocumented features.
• At the end of each session, ask Claude Code: "Document learnings. What failed? What worked? What's next?"
• Have Claude Code write its own prompts by having it launch sub-agents and create common commands in .claude/commands/.
• Symlink CLAUDE.md, AGENTS.md and GEMINI.md into a CONVENTIONS.md
• Prefer creating tools / writing scripts to analyze data and feed results -- reduces input tokens.

• Run with OpenRouter via OPENAI_BASE_URL=https://openrouter.ai/api/v1 OPENAI_API_KEY=$OPENROUTER_API_KEY OPENAI_MODEL=qwen/qwen3-coder npx -y @qwen-code/qwen-code
• Quality: not as good as Claude Code. When prompted to Move AI Image Chat position in tools.json AND in README.md to just below Daydream. Add a small filled-circle icon before "Created: ..." date. The color should be based on how old the created date was. Use primary if it's within the last week, success if it's in the last 30 days, warning if it's in the last 365 day and light otherwise. Also, add a col-xl-3 to the tools-grid cells
  • qwen-code + qwen-coder cost 8 cents and made 3 mistakes.
    • Copied instead of moving the demo
    • Did not render a filled-circle icon. It created an empty badge that ended up not being displayed
    • Did not add a col-xl-3 to the tools-grid cells
  • qwen-code + claude-sonnet-4 cost 104 cents and made no mistakes
  • claude-code cost 29 cents and made no mistakes

• mind-elixir is another option that lets you edit mindmaps and serialize in its own format
• jsmind is yet another but docs are in Chinese
• elkjs seems a good option for laying out nodes in an architecture-style flow diagram

To keep Claude Code in line on my project, I’ve relied heavily on linters, build scripts, formatters, and git commit hooks. It’s pretty easy to get Claude Code to commit often by including it in your CLAUDE.md, but it often likes to ignore other commands like “make sure the build doesn’t fail” and “fix any failing tests”. All my projects have a .git/hooks/pre-commit script that enforces project standards. The hook works really well to keep things in line.

copy(
  $$("table[aria-labelledby=\"version-history\"] tbody tr")
    .map((tr) => {
      const a = tr.querySelector("a");
      const date = new Date(tr.querySelector("time").getAttribute("datetime"))
        .toLocaleDateString("en-GB", {
          day: "numeric",
          month: "short",
          year: "numeric",
        });
      return `- [${a.textContent.trim()}](https://npmjs.com${
        a.getAttribute("href")
      }): ${date}.`;
    })
    .join("\n"),
);

• Increase clarity, providing step-by-step instructions.
• Resolve conflicting instructions.
• Expand instructions to cover all scenarios and edge cases.

• Claude 4 writes exhaustive professionally styled code but struggles over long conversations.
• Gemini 2.5 Pro produces working but “spaghetti” code.
• GPT 4.1 is fast and good, the go-to for usual coding tasks.
• Claude easily swings toward your style but Gemini is stubborn.
• GPT models tend to hallucinate more on bigger tasks.

• O3 is better at finding bugs than Jules, which tends to try and fix them rather than discover them.
• Codex writes more minimal edits in PRs than Jules, which is more verbose.
• Claude Code remains the best at faithfully creating and updating front-end apps.

• GitHub Releases. 2 GiB per file, unlimited total & bandwidth. 🟢 Immortal URL, versioning, easy CI publish. 🔴 Each file must stay < 2 GiB; no built-in SQL.
• Zenodo (CERN). 50 GB per record; one-off bumps to 200 GB. 🟢 DOI assignment, archival mandate. 🔴 Occasional throttled bandwidth; no API for partial file reads.
• Hugging Face Hub. 300 GB per repo; 50 GB per file. 🟢 Git-based, dataset tooling, lively ML community. 🔴 Large files need git-LFS; pushes via LFS can be slow.
• Cloudflare R2. 10 GB storage & 1 M ops / month. 🟢 S3 API, zero-egress to Cloudflare Workers, fast. 🔴 10 GB cap below your 50 GB target.
• Kaggle Datasets. 20 GB per dataset, public only. 🟢 Built-in notebooks & GPU. 🔴 No programmatic SQL API; quotas sometimes change.
• data.world (free). 1 GB total, 100 MB per dataset. 🟢 Nice social features. 🔴 Too small for your size.

• --no-highlight skips code-highlighting. --highlight=pygments adds Pygments styling
• --wrap=none doesn't wrap the content in a single block
• --number-sections adds section numbering (<h2>1. Introduction</h2>)
• --shift-heading-level-by=NUM – shift all headings by NUM levels (e.g., start at <h2> instead of <h1>)
• pandoc -f markdown-auto_identifiers drops the auto-identifiers extension that generates id=... for each heading
• pandoc -f gfm uses GitHub flavored Markdown. Run pandoc --list-extensions=gfm to identify the extensions it uses.
• Pandoc's Markdown extension examples are quite extensive.
• Auto-enabled GFM extensions:
  • alerts: GitHub-style callouts (info, tip, warning) via > [!TYPE] blocks.
  • autolink_bare_uris: Turns bare URLs into links, without needing <...>.
  • emoji: Parses :smile:-style codes into Unicode emoji characters.
  • footnotes: Enables footnote syntax with [^id] and definitions at the bottom.
  • gfm_auto_identifiers: Uses GitHub’s heading-ID algorithm: spaces → dashes, lowercase, removes punctuation.
  • pipe_tables: Enables table.
  • raw_html: Raw HTML is unchanged.
  • strikeout: Enables strikethrough with ~~text~~.
  • task_lists: Parses - [ ] and - [x] items as checkboxes.
  • yaml_metadata_block: YAML front matter for document metadata, e.g. <title>
• GFM extensions worth enabling:
  • ascii_identifiers: Strips accents/non-Latin letters in automatically generated IDs.
  • bracketed_spans: [Warning]{.alert} becomes <span class="alert">
  • definition_lists: Term\n: Definition text becomes a definition list
  • fenced_divs: ::: {.note} block creates a <div class="note">...</div>
  • implicit_figures: Standalone images become <figure> with <figcaption>.
  • implicit_header_references: [Section] is treated as [Section][#section]
  • raw_attribute: <b>bold</b>{=html} is inserted as HTML
  • smart: Converts straight quotes to curly, -- to en-dash, --- to em-dash, ... to ellipsis.
  • subscript & superscript: E.g. H~2~O and E = mc^2^

• You can't run commands. Only prompts. You need to exit codex to run commands. So you can't use it like a shell, e.g. like Warp.dev.
• It doesn't index local code. It runs commands to figure out stuff.
• Code diffs and applying changes are clunky.
• The output is hard to read with text scrolling.
• codex.md can only handle 32K.

• GitHub Copilot can identify the source of a code snippet as a repo. That helps with copyright issues.
• Cursor uses a shadow workspace - a temporary sandbox where it edits files before applying changes at one shot.
• Cursor auto-complete has context of other files, i.e. inserting an class in a .js file based on another HTML file's contents.
• Windsurf seems to be best for large code bases and for large-scale refactoring. It can also run test results fix them.
• Windsurf includes a browser and lets you click on an element and prompt to change its behavior, etc. That's good for front-end developers.
• Roo Code can run scripts as part of the workflow, letting you run linting, tests, starting web apps, query databases, etc.
• Roo Code lets you create persona, e.g. code reviewer, data storytelling and analysis, etc. with access to different tools and behaviors.
• Roo Code does not support auto-complete.

• Recurse center (Brooklyn, online) is a 6/12 week free self-driven programmer retreat. Runs every 6 weeks. You can do whatever you pick. There are daily standups for accountability. The groups are diverse. You can pair with them, pivot ideas, whatever. Principles: push yourself & learn.
• Western education techniques (e.g. spaced repetition, adaptive learning) are very much present in Indian coaching systems, though not known by those names. However, interventions are hard since class 12 students just don't have enough time.
• Coaching classes are a social phenomenon. It's not the smart students who are pulling in their friends. Smart students actually follow the popular students. (Coaching classes are below the typical smart students' standards.)
• Monetizing coaching is hard. People don't want to pay for advice, and welcome free advice only if they ask for it. Coupling with execution is necessary.
• Aider's integrations make it more powerful than Cursor/Windsurf. It auto-lints, runs test cases. Allows different models for "architecting" (generating changes) vs "editing" (applying code). It reads from the screen logs. Context is manual, not automated. Uses an ai! comment to trigger changes and ai? to ask questions.
• Cline.bot is another Cursor-like open source AI code editor that's a VS Code plugin.
• When coding with LLMs, a useful workflow is: data schema ➡️ interfaces ➡️ LLM-generated test cases ➡️ code.
• ShellSage is a tmux based LLM tool for the command line. It screen-grabs from tmux, which is powerful.
• Some MCPs that have proven useful: vega-lite, SQLite, sequential thinking, memory
• make sucks but is hard to beat. just comes closest.
• CRDTs are more powerful than for just collaborative editing. It can power a peer-to-peer Internet (beginning with office tools). Versioning schema is still problematic. yjs is a good start but automerge (Rust, WASM) is faster and may be better.
• Fermyon hosts WASM serverless functions.

• 🔴🟡🟢 for low/medium/high priority
• ⭐️ or ❤️ or 👍 for ratings or emphasis
• ✅ for completed tasks
• 💡 for ideas
• ⚠️ or ❗️ for warnings / issues

• I have 3 categories. Things I learnt, which I just note. Things to explore, which I can delegate, defer, drop, or do at any time. Things to do, which are the hardest and pile up.

• Useful for: Document conversion, Thumbnail generation, Text extraction, Merging / splitting documents

• Snowman is a browser-based Twine 2 story template format.
• These enable behavioural experimentation. Cheaper than using tools like Gorilla.sc and Pavlovia for behavioral experiments
• For example, you can present a social or political issue and see if people change their opinions more or less depending on the content/path they see. Or, if it varies by demographics. Or, check if repeated mentions or emotional hooks improve memory / retention. More research ideas

• The VS Code plugin makes development very easy
• Marp CLI makes deployment easy.
• I used it for my talk on LLM Hallucinations (source).
• Supports all bespoke features and plugins
• Transitions. Requires OS animation effects to be enabled
• Animated SVG backgrounds are a good add-on.

1. Different URLs. E.g. https://example.org/about/ vs https://example.org/about.md (this is how Jekyll or Hugo work). Use for static sites generators.
2. JavaScript. Inject after Markdown: <script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script><script>document.body.innerHTML = marked(document.body.textContent);</script>. Use for dynamically generated static sites.
3. URL query parameters. E.g. ?format=markdown vs ?format=html vs ?format=json. Use in APIs.
4. Content Negotiation. Based on the user agent and Accept header, serve Markdown or HTML. Send Vary: Accept to indicate that the response depends on the Accept header. Use for dynamic web apps.

• Capture: Write down every idea or piece of information on a separate note. Use your own words to ensure understanding.
• Organize: Consolidate fleeting notes into permanent ones. Assign unique identifiers to each note for easy reference.
• Connect: Link related notes to form a web of knowledge. This can be done with tags, references, or hyperlinks in digital systems.
• Review: Regularly revisit your notes to strengthen connections and discover new insights.

• Copy an entire issue (with history) from GitHub, Gitlab, or JIRA
• Copy an entire PR (with code changes) from GitHub, Gitlab, or Bitbucket
• Copy CI/CD logs from GitHub Actions, Gitlab CI, Azure DevOps, etc.
• Copy entire conversation thread in Gmail or Discourse, Service now etc.
• Copy product reviews from Amazon, Shopify, etc.
• Copy page(s) from wikis and content sites like Wikipedia, StackOverflow, etc.
• Copy survey responses from Google Forms, Typeform, etc.
• Copy all interactions with a contact (including interactions, proposal history) from HubSpot or Salesforce
• Copy transcripts from Zoom, Teams, Google Meet, etc.
• Copy as Markdown from Word, GDocs, PDF or HTML
• Copy the summary of an analysis as well as all key metrics from any dashboard
• Copy SAP invoices
• Copy JDs, CVs, and reviews from Workday, BambooHR, DarwinBox, etc.
• Copy design specs, component libraries, and style guides from Figma, Miro, etc.
• Generated with the help of ChatGPT -- link not working

• Marker from DataLab seems comparable but is CC-BY-NC-SA.
• MinerU convert medical textbooks to Markdown well.
• Gemini Flash may be more cost effective and better

• Autocomplete and tab completion works perfectly, out-of-box.
• Syntax highlighting is beautiful
• Great multi-line editing

• Redash feels too clunky / enterprise-y rather than open-source-y.

• uvx files-to-prompt
• npx git-ingest
• ingest - written in Go, only Mac/Linux binaries

• PyMuPDF4LLM uses MuPDF. Requires PyTorch.
  • PYTHONUTF8=1 uv run --with pymupdf4llm python -c 'import pymupdf4llm; h = open("pymupdf4llm.md", "w"); h.write(pymupdf4llm.to_markdown("$FILE.pdf"))'
• markitdown from Microsoft. PDF via PDFMiner, DOCX via Mammoth, XLSX via Pandas, PPTX via Python-PPTD, ZIP, etc.
  • PYTHONUTF8=1 uvx markitdown $FILE.pdf > markitdown.md
• Docling by IBM. Unable to install via pip on Windows AND on Linux.
• MegaParse uses libreoffice, pandoc, tesseract-ocr, etc. Requires OpenAI API key.

• Applicable to well structured documents, e.g. technical books, manuals, etc.
• Create a hierarchical outline of the document. Code
  • Keep the top-level headings.
  • Preserve the first ~100 characters of opening text from each section.
  • Present the second-level headings, but without any subsidiary content.
  • Provide each section a unique 8 digit hex identifier.
  • Each section heading is followed by a guiding comment for the model: Section collapsed - expand with expand_section("{identifier}").
• Then read the relevant sections as context to answer the question. Code

• Jekyll - Ruby - 2008
• MkDocs - Python - 2014
• GitBook - JavaScript (Node.js) - 2014
• MkDocs Material - Python (MkDocs-based) - 2016
• Docsify - JavaScript - 2016
• MdBook - Rust - 2017
• Antora - JavaScript (Node.js) - 2017
• Docusaurus - JavaScript (React) - 2017
• JupyterBook - Python - 2019
• Keenwrite - Java - ~2019
• Honkit - JavaScript (GitBook fork) - 2019
• Nextra - JavaScript (Next.js) - 2020
• Astro - JavaScript/TypeScript - 2021
• Hugo Book - Go (Hugo-based) - ~2020
• Clowncar - JavaScript/Node.js - ~2021
• Quarto - R and Python - 2022
• Starlight - JavaScript/TypeScript - 2023

• Mentioning examples and schema and "JSON" helps. When providing examples, using (user, assistant) message pairs helps (I think it's because it's easier for the LLM to parse).
• Using a {reasoning, answer} schema (with reasoning first) helps. Make reasoning concise and relevant Ref Arxiv
• We already know code in JSON is not a great idea. Ref

• Jina AI Meta Prompt is an example
• Remotion system prompt is an example
• https://docs.fastht.ml/llms-ctx.txt
• https://docs.fastht.ml/llms-ctx-full.txt

• Publishing any notebook to static.marimo.app as a static app
• Creating a SINGLE link that embeds the ENTIRE notebook in the URL!
• Runnable via uvx marimo edit

• This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two:
  • the 2.0 programmers (data labelers) edit and grow the datasets, while
  • a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces.
• Adaptive UI ideas:
  • Adaptive Fields: Show only required fields based on what the user field so far.
  • Smart Inputs: Dropdowns and auto-complete based on user's context.
  • Smart Themes: Change font size, contrast, theme guessing the user's age and preferences.
  • Dynamic Menus: Show what they might need to do next. Like Nokia's right button, but using LLMs.
  • Smart Tooltips: Check what the user's doing (delays, confusions, previous clicks, current actions) and show relevant tips.
  • Personalized Layout: Show only the relevant sections of the app. E.g. based on what they're doing.
  • Smart Charts: Create the right chart that solve the user's question.
• Adaptive Back-end
  • Dynamic APIs: Create endpoints on the fly based on user needs
  • Dynamic Indexing: Create & update indices on the fly based on user needs
  • Dynamic Schema: Create & update schema on the fly based on user needs
  • Dynamic Migration: Migrate to a new database or OS or language as required
  • Dynamic Queries: Create SQL/NoSQL queries to solve the user problem
  • Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required
  • Dynamic Logging. Log what's required. Explain why it's logged and what's happening. Fix code that raised the error
  • Dynamic Caching. Cache what's likely to be required. Evict what may not be required. Figure out cache keys.

• Vaishnavi and Ranjeet edited code
• Indal and Koustav didn't

• 04 Oct 2024. PDF-Extract-Kit does PDF layout, formula, table, and OCR extraction using various models.
• 04 Oct 2024. llmsherpa extracts PDF layout, tables, not OCR

• Claude Artifacts Prompt
• Val.Townie system prompt. Good example of how to create
• Cursor editing
• Cursor debugging
• Cursor conversation

• Claude
• OpenAI
• Gemini

• Cognition
• Factory
• Codegen

• 3D Printable Models:
  • STL (Stereolithography): Defines the surface geometry of 3D objects using triangular facets.
  • OBJ (Wavefront OBJ): Describes 3D geometry including vertices, textures, and normals.
  • X3D: An XML-based file format for representing 3D computer graphics.
• Vector Graphics:
  • SVG (Scalable Vector Graphics): Defines vector-based graphics in XML format, useful for illustrations, diagrams, and user interface elements.
• CAD Drawings:
  • DXF (Drawing Exchange Format): Represents CAD data, including shapes, lines, and curves, used in engineering and architecture.
• Circuit Designs:
  • KiCAD: An open-source software suite for Electronic Design Automation (EDA), which uses various file formats like PCBNew and EESchema to represent circuit designs.
• Blueprints and Architectural Designs:
  • GML (Geography Markup Language): Encodes geographical features and spatial information.
  • CityGML: A specific GML application schema for modeling and exchanging 3D city models.
• Molecular Structures:
  • PDB (Protein Data Bank): Describes the three-dimensional structures of molecules.
  • CML (Chemical Markup Language): An XML-based standard for representing molecular data.
• Robotics and Automation:
  • URDF (Unified Robot Description Format): Defines the physical configuration of a robot, including joints, links, and sensors.
  • COLLADA (Collaborative Design Activity): An XML-based schema to describe digital assets for 3D applications, often used in robotics.
• Geospatial Data:
  • KML (Keyhole Markup Language): Used for geographic data visualization, primarily in Google Earth.
  • GeoJSON: A format for encoding a variety of geographic data structures using JSON.
• Mathematical Markup:
  • MathML (Mathematical Markup Language): Describes mathematical notation and captures both its structure and content.
• Music and Sound:
  • MusicXML: Encodes sheet music in a structured format that can be easily shared between different music notation software.
• Documents and Text:
  • DocBook: A semantic markup language for technical documentation.
  • Markdown: A lightweight markup language with plain text formatting syntax.
• Biological Data:
  • SBML (Systems Biology Markup Language): Represents computational models of biological processes.
  • PhyloXML: An XML format for representing phylogenetic trees.
• Game Development:
  • FBX (Filmbox): A file format for 3D animation that can hold information about the geometry, textures, and animations.
  • VRML (Virtual Reality Modeling Language): Describes interactive 3D objects and worlds.
• Data Visualization:
  • ChartML: Encodes charts and graphs in a structured format.
  • D3.js (Data-Driven Documents): Uses HTML, SVG, and CSS to bring data to life with interactive visualizations.
• Building Information Modeling (BIM):
  • IFC (Industry Foundation Classes): Describes building and construction data.
• Textiles and Fabrics:
  • LoomML: Represents the design and structure of woven fabrics.
• Augmented Reality and Virtual Reality:
  • ARML (Augmented Reality Markup Language): Defines how augmented reality applications should behave and what content they should display.
  • VRML (Virtual Reality Modeling Language): For describing interactive 3D objects and worlds.
• Medical Imaging and Health Data:
  • DICOM (Digital Imaging and Communications in Medicine): Encodes medical imaging data.
  • HL7 (Health Level 7): A set of standards for the exchange of information between medical applications.
• Simulation Data:
  • FMI (Functional Mock-up Interface): Represents and exchanges dynamic simulation models.
  • SBML (Systems Biology Markup Language): For computational models of biological processes.
• Sound and Audio:
  • MML (Music Markup Language): For encoding music notation and performance information.
  • SoundFont: A file format for defining musical instrument sounds.
• Animation and Visual Effects:
  • BVH (Biovision Hierarchy): Encodes motion capture data.
  • Alembic: A computer graphics interchange framework primarily for exchanging animation and visual effects data.
• Textile Patterns:
  • WIF (Weaving Information File): Describes weaving patterns and structures.
  • Knitting Markup Language: Encodes knitting patterns in a structured format.
• Scientific Data:
  • CDF (Common Data Format): Used for storing scientific data.
  • NetCDF (Network Common Data Form): Supports the creation, access, and sharing of array-oriented scientific data.
• Photography and Imaging:
  • XMP (Extensible Metadata Platform): Used for embedding metadata in digital images and other media files.
• Construction and Engineering:
  • LandXML: For civil engineering and land surveying data.
  • gbXML (Green Building XML): Facilitates the transfer of building data for analysis of energy and environmental performance.
• Packaging and Retail:
  • BPL (Barcode Product Labeling): Encodes information for product packaging and labeling.
  • GS1 XML: Used for electronic business messaging, including product identification and tracking.
• Typography and Font Design:
  • UFO (Unified Font Object): A format for storing font data.
  • SFNT (Spline Font): Encodes scalable font information.
• Product Data Management:
  • PLMXML (Product Lifecycle Management XML): Used for sharing product data across PLM systems.

• Always create a new venv via VS Code when starting a training session. Helps reproduce issues (though I could use Colab instead)
• Create an empty .ipynb notebook and double-click it. That's another way (though slower) to open a Jupyter notebook

• Select any code and command Explain this to understand the code
• %something in command bar searches ACROSS files for a term. Exactly like Ctrl+Shift+F
• Copilot has an Inline Chat: Start in Terminal (that needed me to unbind Ctrl+I in bash to work)
• Ctrl+2 opens a second window on the side. Ctrl+1 goes back to the first window
• Terminal: Open Detected Link lets you scroll through detected (file) links in terminal
• Terminal sticky scroll is transparent. (But Terminal stick scroll isn't working for me.)
• Copilot uses last 10 commit messages, Jupyter notebook kernel state (variables) as additional context
• 1.88: supports locked scrolling to sync scrolling of side-by-side windows

• Why LLMs can't be used for spelling
• Why LLMs are better at English than other languages
• Why LLMs are bad at math
• Why SolidGoldMagiKarp is a single token
• Why trailing spaces are bad
• Why YAML tokenizes more efficiently than JSON

• Example of fine-tuning Mistral. Consumed 28 computes ($2.8)
• Axlotl is what the top fine-tuned LLMs are trained on
• Deepspeed provides distributed training
• Flash attention lets data stay on GPU
• Sample packing packs samples of different lengths into equal length tensors

• Visualizing partial election results
• D3 Force Dilemmas: Data Distortion