ML engineer vs software engineer: the real difference
An ML engineer ships models in production; a software engineer ships systems. The skills overlap more than the titles suggest, and the move between them is shorter than you think.
By the roles.cc team··10 min read
The short version: a software engineer ships systems that behave the way they were written to behave, and an ML engineer ships systems whose behavior is learned from data and is never fully deterministic. Both write production code, own services, and get paged at 2am. The difference is what they are debugging at 2am. A SWE is chasing a logic error or a bad deploy. An ML engineer is asking whether the model regressed, whether the training data drifted, or whether a feature pipeline started sending garbage. Same craft, different failure modes.
This post is about the real difference across four things people actually care about: skills, interviews, comp, and demand. Then how to move between the two roles, which is shorter than the titles suggest. It is not about ML versus data science (a separate line) or about research scientists who publish. For the broader market read, see the AI startup engineering job market and what LLM engineering jobs actually want.
What does an ML engineer actually do that a software engineer does not?
Most ML engineering is still software engineering. Job postings keep landing near the same ratio: roughly 70 percent of the work is building and operating systems, and 30 percent is the modeling. The real difference lives in that 30 percent, plus the operational tail it creates.
- The artifact is data, not just code. An ML engineer versions datasets and model weights, not only source. A change can come from a new training set with zero code diff. That breaks a lot of normal SWE intuition about what "deploying" means.
- Correctness is statistical. A SWE asks: is this output right. An ML engineer asks: is this output right often enough, and how do I measure that. Evaluation harnesses, offline metrics, and online A/B tests are first-class work, not an afterthought.
- The system degrades on its own. A web service that is left alone keeps working. A model left alone slowly rots as the world drifts away from its training data. Monitoring for drift and retraining is a permanent chore with no SWE equivalent.
- The toolchain is heavier. Feature stores, vector databases, GPU scheduling, experiment tracking, and serving frameworks sit on top of the normal stack. The platform surface is bigger, which is why platform and infrastructure engineering overlaps with ML infra so often.
What does not differ: both roles live in Git, ship through CI, write tests, review pull requests, and answer for latency and cost. If you picture an ML engineer as a person staring at math all day, you are picturing a research scientist. The ML engineer is the person who turns that research into a service that stays up.
What skills separate the two?
Start from a shared base, then add. Both need strong programming (Python and one systems language are common), data structures, distributed systems sense, and the ability to own a service end to end. The ML-specific layer is narrower than people fear.
| Area | Software engineer | ML engineer (the added layer) |
|---|---|---|
| Core | APIs, services, databases, concurrency, system design | All of that, plus the items below |
| Math | Light. Big-O, some discrete math | Linear algebra, probability, gradient intuition. You do not derive proofs, you read papers and reproduce them |
| Data | Schemas, queries, caching | Feature engineering, labeling, train/test splits, leakage, class imbalance |
| Modeling | Rare | Knowing when to fine-tune vs prompt vs train from scratch, and reading an evaluation curve honestly |
| Ops | Deploys, on-call, observability | Model monitoring, drift detection, retraining pipelines, GPU cost |
The added layer is real but learnable in months, not years, if you already ship production software.
Two practical notes. First, the LLM era shifted a lot of ML engineering away from training models and toward orchestrating them: retrieval, prompting, evals, and serving. That work looks much more like normal backend engineering, which is exactly why SWEs are moving into it fast. Second, the math bar for an applied ML engineer is lower than the bar for a research scientist by a wide margin. You need to understand what a model is doing, not invent a new one.
How do the interviews differ?
Both loops share a coding round and a behavioral round. After that they fork. Knowing the fork in advance is half the battle.
| Round | SWE loop | ML engineer loop |
|---|---|---|
| Coding | Data structures and algorithms | Same, sometimes lighter, often with a data-manipulation problem (pandas, numpy) |
| System design | Design a scalable web system | Design an ML system: training pipeline, feature store, serving, monitoring, retraining |
| Domain | Rare | ML fundamentals: bias/variance, overfitting, metrics (precision, recall, AUC), when to use which model |
| Practical | Take-home or pairing | Sometimes a modeling take-home or a debugging-a-bad-model exercise |
At startups the ML loop is often shorter and more applied than at large labs.
The ML system design round is the one that trips up SWEs. It is not harder, it is different. You are asked to reason about data freshness, training/serving skew, offline vs online evaluation, and how you would catch a silent quality regression. If you have shipped a real ML feature, you can answer it. If you have only read about ML, you cannot fake it. For the SWE side of design prep, see startup system design interview prep, and for the take-home question, are take-home assignments worth it.
Who gets paid more, the ML engineer or the software engineer?
At the same level and company, ML engineers usually carry a premium over generalist SWEs, but it is smaller than the internet implies. The honest range is a 5 to 20 percent base premium at most startups, widening at companies whose product is the model. The bigger swing is which company you join, not which title you hold.
5 to 20%
typical ML base premium
over a generalist SWE at the same level
$190K to $230K
senior ML eng base, SF/NYC
illustrative 2026 range, not advice
Equity
where the real spread is
stage and company matter more than title
Worked example (illustrative, not advice). Take a senior engineer at a Series A startup in San Francisco. A generalist SWE offer might be $185,000 base. The ML engineer offer at the same company and level might be $205,000 base, roughly an 11 percent premium, with similar equity. Now compare that ML engineer offer to a senior SWE at a company that just raised a large round: the SWE could clear the ML engineer on total comp purely on equity and stage. Title is a weak comp lever. Stage and ownership are strong ones. We sort the board and recent raises by funding recency for exactly this reason. For the underlying numbers, see senior software engineer salary in SF and NYC for 2026.
Which role has more demand right now?
Both are in demand, and the two are converging at the LLM application layer. Generalist SWE remains the larger market by headcount: every company needs them, and the openings on the board reflect that. ML and AI engineering is the faster-growing slice, concentrated at AI-first startups and at any company building features on top of foundation models.
The useful nuance: "ML engineer" now spans two very different jobs. One is classic ML (recommendations, fraud, search ranking) that needs real modeling depth. The other is LLM application engineering (retrieval, agents, evals, serving) that needs mostly strong backend skills plus a working mental model of how these systems fail. The second category is hiring aggressively and is the easiest on-ramp for a SWE. See which startup sectors are hiring engineers for where the roles cluster.
How do you move from software engineer to ML engineer (and back)?
The SWE-to-ML move is the more common direction and the more practical one, because you already have the 70 percent. You are buying the 30 percent, not the whole job. A focused path:
- 01Ship one real ML feature inside your current SWE job. A search ranking tweak, a classifier, a recommendation, or an LLM-backed feature. Volunteer for the ML-adjacent work. This single line on your resume beats any course.
- 02Own the serving and monitoring, not just the model. This is where SWEs have a natural edge and where ML teams are perpetually short-handed. You become the person who makes the model reliable.
- 03Learn the evaluation discipline. Be able to read a precision/recall tradeoff, design an offline eval, and reason about why a model that looks good offline degrades online. This is the skill interviewers probe.
- 04Build the system-design vocabulary. Practice designing an ML system out loud: data, features, training, serving, monitoring, retraining. You can carry your existing distributed-systems intuition straight into it.
- 05Target the LLM application layer first. It is the shortest jump from backend SWE and the fastest-hiring segment. Classic deep-modeling roles can come later if you want them.
The reverse move (ML to general SWE) is easy and rarely needed: an ML engineer already writes production software. The trap to avoid is becoming an ML engineer who cannot ship anything outside a notebook. That person struggles in both markets. Stay a strong software engineer first. For the wider question of picking a track, see backend, frontend, and fullstack career paths and how to become a staff engineer.
An ML engineer who cannot ship outside a notebook is weak in both markets. A SWE who can ship a model in production is strong in both.
Which one should you aim for?
Pick by what you want to debug at 2am, not by the comp premium. If you like deterministic systems, clear correctness, and broad optionality across every company, stay a generalist SWE. If you are drawn to fuzzy correctness, evaluation, and the specific puzzle of keeping a learned system honest in production, lean ML. The skills overlap enough that you are not locking a door either way. You are choosing which 30 percent you want to specialize in.
Questions people ask
What is the difference between an ML engineer and a software engineer?
A software engineer builds systems that behave exactly as written, while an ML engineer builds systems whose behavior is learned from data and is never fully deterministic. Both write production code and own services, but the ML engineer also versions datasets and models, measures correctness statistically, and monitors for drift over time. Roughly 70 percent of ML engineering is ordinary software engineering, with about 30 percent being the modeling and evaluation layer on top.
Do ML engineers get paid more than software engineers?
At the same level and company, ML engineers usually carry a 5 to 20 percent base premium over generalist software engineers, which is smaller than most people assume. The larger swing in total compensation comes from which company and stage you join, driven mostly by equity, not by the title. A senior SWE at a company that just raised can out-earn an ML engineer at an earlier-stage one.
Is it hard to switch from software engineer to ML engineer?
It is more of a focused jump than a full career change, because a SWE already has about 70 percent of the job. The fastest path is to ship one real ML feature inside your current role, own its serving and monitoring, and learn the evaluation discipline. Targeting LLM application work first (retrieval, evals, serving) is the shortest on-ramp because it relies mostly on strong backend skills.
How is an ML engineer interview different from a SWE interview?
Both share a coding round and a behavioral round, then fork. The ML loop adds a machine learning system design round (training pipeline, feature store, serving, monitoring, retraining) and an ML fundamentals round covering metrics, overfitting, and model choice. The ML system design round is the one that most often trips up software engineers, because it requires reasoning about data freshness and silent quality regressions rather than just scaling a web service.
Do you need a strong math background to be an ML engineer?
For an applied ML engineer, the math bar is much lower than for a research scientist. You need linear algebra, probability, and enough gradient intuition to read a paper and reproduce a model, not to invent one. Most of the day-to-day is software engineering, data work, and evaluation, so strong programming and systems skills matter more than advanced math.
Which has more demand, ML engineering or general software engineering?
General software engineering is the larger market by headcount because every company needs it, while ML and AI engineering is the faster-growing slice concentrated at AI-first startups. The two are converging at the LLM application layer, where retrieval, evals, and serving look much like normal backend work. That convergence makes LLM application roles the easiest on-ramp for a software engineer wanting to move into ML.
Put the signal to work
The board lists live roles at startups that just raised, free and unfiltered. Or drop your CV and we bring the right ones to you.
About roles.cc. roles.cc is a recruiting agency for software engineers at venture-backed startups in San Francisco, New York, and other major US hubs. The public board lists engineering roles pulled straight from each company's own job site, sorted by how recently the company raised. It is free for engineers. Start with the live board or what we do.