Key Points

What Is Agentic Coding?

  • Agentic coding: AI agents plan, write, test, debug, and revise code with limited human intervention. An agent acts; a chatbot advises.
  • Agent = LLM + harness + tools + agent loop. The model reasons; the harness supplies files, a terminal, tests, permissions, and memory.
  • Requests range from a single edit to a whole project. More autonomy shifts effort from approving actions to specifying intent in advance and reviewing results afterwards.
  • The leading tools are within a point or two of each other. Use what you have access to, and learn the principles.

Safety and Security: Limit What the Agent Can Access

  • Institutional data policies apply unchanged: sensitive or restricted data stays away from unvetted AI services, on every route.
  • Agents run with your permissions and scan your workspace for context. Assume anything on disk in plaintext can be read.
  • Prompt injection is the threat: anything the agent reads can carry instructions. Treat it all as untrusted, keep the web session’s firewall on, and review the pull request before merging.
  • Six limits determine how much damage injection can do: network allowlist, cloud VM, command rules, password-manager credentials, branch plus pull request, and whom you trust.
  • Instructions influence behavior; permissions constrain it. A rules file is text the model weighs, not an ability it lacks.
  • The only secret an agent cannot leak is one that is not there. Load keys at runtime with op read.
  • First session: a cloud route with plan mode, confirm the session is cloud-hosted, a branch named for you, no .env.

Trust: Packages, Models, and Providers

  • Agents install packages and download models quickly and in bulk. Keep those decisions on the list that requires approval.
  • Hallucinated package names are an attack vector (slopsquatting). Verify that a package exists and is official before installing anything an agent suggests.
  • Model weights in pickle formats are executable code. Prefer safetensors, download from verified organizations, do not enable trust_remote_code by default, and check provenance.
  • Backdoors can be in the weights themselves (poisoned training data, surgical edits, trigger behaviors). No scanner or benchmark detects them; provenance and reviewing model output are the defenses.
  • The agent tooling is itself an attack surface. Keep it updated, and never point an agent at production or irreplaceable data.
  • Vet the provider’s data policy: training default, retention, jurisdiction, and whether an institutional agreement applies.
  • Provider defaults differ by plan tier. Consumer Claude and Copilot plans train on your data unless you opt out; team, enterprise, API, and education tiers do not. Check the setting, not the brand.
  • Each rule here has a named incident behind it: torchtriton, Ultralytics, huggingface-cli, Amazon Q, Replit, DeepSeek, Samsung, the fake OpenAI repository, InstructLab. None required a sophisticated attacker; all required a missing check.

Planning with Agents

  • Plan before code. Planning measurably improves accuracy, and a good plan usually reduces token use. Keep planning proportional to the task.
  • A plan is only as good as its context. Provide the goal, constraints, compute, existing code, standards, and prior decisions; for long work, keep the plan in a file the agent updates.
  • Agree team conventions first (branches rather than forks, pull-request size, who reviews, what the agent may not modify, where new files go) and commit them as a CONTRIBUTING.md that agents also read.
  • Use plan mode (read-only) to review a plan. The deliverable is a plan.md with ordered features and a check for each, committed before any code.
  • Start from a minimum viable pipeline: a slice of data, one model, something you understand end to end.
  • Question AI design suggestions before adopting them, most carefully where your domain knowledge is weakest. If you cannot explain it, you do not yet own it.
  • Write the project context file (CLAUDE.md, AGENTS.md, copilot-instructions.md) at the start of the project. Keep it short, operational, and backed by permissions where it matters.

Feature-Based Development and Good Prompting

  • The agent is not a magic wand. A person who knows the domain decides what to build next; the agent builds that one thing.
  • A feature is one thing you can check. Work through the plan one feature per session, with a small diff and one pull request.
  • Underspecified prompts receive average-case answers, and research data is rarely the average case. The failure mode is plausible-but-wrong.
  • Context engineering sets the standing rules (context file, standards, plan); prompt engineering phrases the specific request. Use both.
  • Good prompts provide evidence, ask for a diagnosis before a change, and specify inputs, output, the completion check, and what not to modify.
  • Maintenance tasks (refactors, docstrings, README audits, environment pins, merge summaries) are well suited to agents under the same small-specified-checkable rules.
  • Always ask the agent to list the choices you did not specify.

Verification and Testing: No Escaping Good Data Science

  • Checking is the bottleneck. Agents multiply commits far more than releases, pull requests are larger and harder to review, and unreviewed merges increase. Verification is the scarce skill.
  • The pull request is the final human check. Review by eye alone does not scale, so it needs tests on every push and checks that fail loudly.
  • Established practice still applies: know your data, compare to a source of truth, know what your model responds to, reproduce the result, plot and review.
  • Test-driven development reverses the usual order: write the test that states the required behavior first, then have the agent make it pass. The test is the specification.
  • For research code, the tests to write first are about the data and the result: data expectations, a hand-checked subset, seed stability, sensible top predictors, no leakage. Each can be a test the agent writes and CI runs.
  • Split by the unit that repeats, and assert it. A clean-running 0.91 can be 0.58 on unseen subjects.
  • After every feature, before the next: have the agent list its assumptions, propose tests and edge cases, add them, and run them. Then commit.
  • Make verification automatic: tests in CI and a protected main. Autonomy is purchased with verification.
  • Choose a review posture deliberately: the pull request as the gate (the agent commits freely; one full review of the diff) when tests and CI support it; per-change review when they do not. Incoming pull requests on a shared project are untrusted input, and the human makes the merge decision.
  • Leave evidence in the repository and direct the agent to it. It reads figures and metrics, but only those you name, and you check every number it cites.

MCP Tools and Skills: Extending Your Agent

  • API: you write the calling code. MCP: the agent discovers the tools and decides when to call them. Use MCP when the agent should reach a system on its own, and a direct API when you are writing the pipeline.
  • An MCP server exposes tools, resources, and prompts over a standard protocol. Connecting one is comparable to installing a plugin.
  • A skill is a procedure: packaged instructions loaded on demand, with no server and no network. It saves context because it expands only when relevant.
  • Context file, then skills, then hooks (which always run), then MCP: codify a workflow at the lowest level that enforces what you need.
  • Everything an MCP tool returns is untrusted input. Scope tokens to the minimum and configure per project.
  • Vet a skill or MCP server before installing it, regardless of the author. Parse, Matt Pocock’s skills, gstack, Claude Science, and ToolUniverse are reasonable starting points.

Cost, Context, and Energy

  • A chat query is about 0.3 Wh; a median coding session about 41 Wh; a heavy agentic day about a dishwasher cycle. Estimates vary by an order of magnitude, so think in tens to hundreds of watt-hours per session.
  • The issue is aggregate demand, and inference now dominates. Usage patterns are the variable you control.
  • Agentic sessions chain hundreds of model calls, and context accumulation makes late messages far more expensive than early ones.
  • Watch usage, match the model to the task, keep sessions short and compact, use skills, and compare real session cost across models before committing.
  • A specific prompt in a clean context is better, cheaper, and lower in energy. One discipline, three benefits.

What the Research Shows, and Where This Leaves Us

  • Agentic tools amplify expertise. Verification skill, not typing speed, is the limiting factor, and perceived speedups can be illusory, so measure.
  • Learning with AI can cost understanding (17% lower quiz scores, largest gap in debugging). Use the agent to ask why, not only to produce code.
  • How novices build supervisory expertise in an agentic setting is an open and serious question.
  • Agentic coding is another abstraction layer, like the move from assembly to Python, but a non-deterministic one. The craft is making it deterministic enough through specification and verification, without spending more time prompting than writing the code would take.
  • At the feature level the productivity gains are substantial, and they scale with prior expertise. Agents amplify expertise; they are not a magic wand.
  • The work shifts toward specifying intent, designing verification, and exercising judgment, and good data science practice still applies in full.
  • When AI teaches you something at some cost, write it up (blog, notebook, lesson) so the next person starts from the verified version.