Benchmarking ASP.NET Core Applications: BenchmarkDotNet, Load Testing, and Finding Real Bottlenecks

Measure ASP.NET Core performance correctly, separate microbenchmarks from load tests, and optimize the bottlenecks that actually matter

Performance optimization without measurement is mostly guesswork. A method may look expensive but account for almost none of your application’s total response time. A clever optimization may make a benchmark 40 percent faster while having no measurable effect on production.

At the same time, a slow database query, blocked Thread Pool, excessive allocation, or overloaded downstream service may be quietly determining the capacity of the entire system. Benchmarking gives us a disciplined way to separate these situations. In ASP.NET Core, that means knowing when to use BenchmarkDotNet, when to load-test the complete application, which numbers actually matter, and how to move from “the application feels slow” to a specific bottleneck we can prove and fix.

We Have Spent Several Articles Making Things Faster

Over the previous parts of this series, we have explored several performance techniques.

We reduced unnecessary allocations.

We examined garbage collection and object pooling.

We streamed large data instead of buffering everything.

We used pipelines to move data efficiently.

We designed backpressure so overloaded applications fail gracefully instead of collapsing.

Most recently, we explored Native AOT for faster startup and leaner deployment.

Every one of those techniques can improve a system.

But there is a problem.

How do we know that your application needs them?

Consider an endpoint that takes 180 milliseconds:

Request
   ↓
Authentication        3 ms
   ↓
Application logic     5 ms
   ↓
Database query      160 ms
   ↓
Serialization         4 ms
   ↓
Response              8 ms

Suppose we spend two days optimizing serialization.

We make it twice as fast.

Excellent.

Serialization now takes:

2 ms

Total response time becomes roughly:

178 ms

We performed a successful optimization that barely improved the user experience.

That is why performance engineering begins with measurement.

Benchmarking and Load Testing Are Not the Same Thing

These terms are often mixed together, but they answer different questions.

A microbenchmark asks:

How efficiently does this small piece of code execute?

A load test asks:

How does the complete application behave when many users or requests hit it?

A stress test asks:

What happens when we push the system beyond normal operating conditions?

Microsoft makes the same distinction in its official ASP.NET Core load and stress testing guidance. Load testing checks whether an application can satisfy its response goals under a specified expected load. Stress testing deliberately subjects it to abnormal conditions to examine stability and recovery.

These tests complement each other.

They are not interchangeable.

Start With the Question

Before opening a benchmarking tool, define what you are trying to learn.

Bad question:

Is this code fast?

Better question:

Which of these two implementations allocates less memory and completes faster?

Better load-testing question:

Can this API sustain 1,000 requests per second while keeping p95 latency below 250 milliseconds and errors below 0.1 percent?

Good performance tests have a hypothesis.

Without one, you can generate thousands of numbers without learning anything useful.

BenchmarkDotNet Measures Small Pieces Extremely Well

BenchmarkDotNet is widely used in the .NET ecosystem for controlled measurements of small units of .NET code.

Imagine that we have two ways of constructing a response string:

public string BuildWithInterpolation()
{
    return $"{_firstName}:{_lastName}:{_customerId}";
}

and:

public string BuildWithConcat()
{
    return string.Concat(
        _firstName,
        ":",
        _lastName,
        ":",
        _customerId);
}

Which is faster?

You could write:

var sw = Stopwatch.StartNew();

for (var i = 0; i < 1_000_000; i++)
{
    BuildWithInterpolation();
}

sw.Stop();

Console.WriteLine(sw.Elapsed);

But this innocent-looking test has many problems.

What happened during JIT compilation?

Was the machine warming up?

Did garbage collection happen?

Did the runtime optimize something?

Did another process interrupt execution?

Did the compiler eliminate work?

How many allocations occurred?

Was the first execution representative of later executions?

BenchmarkDotNet exists because reliable microbenchmarking is harder than starting and stopping a stopwatch.

A Simple BenchmarkDotNet Benchmark

A basic benchmark might look like this:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

[MemoryDiagnoser]
public class CustomerFormatterBenchmarks
{
    private readonly string _firstName = "Maria";
    private readonly string _lastName = "Silva";
    private readonly int _customerId = 74281;

    [Benchmark(Baseline = true)]
    public string Interpolation()
    {
        return $"{_firstName}:{_lastName}:{_customerId}";
    }

    [Benchmark]
    public string Concat()
    {
        return string.Concat(
            _firstName,
            ":",
            _lastName,
            ":",
            _customerId);
    }
}

BenchmarkRunner.Run<CustomerFormatterBenchmarks>();

BenchmarkDotNet runs repeated measurements and produces statistical results rather than one stopwatch reading.

The important point is not whether Concat or interpolation wins this particular example.

The important point is that we now have a repeatable experiment.

Mean Is Only the Beginning

A typical benchmark report may contain values such as:

Method          Mean      Error     StdDev     Allocated
Interpolation   82 ns     ...       ...        96 B
Concat          74 ns     ...       ...        88 B

Those numbers are illustrative.

Do not copy them into a performance claim.

Run the benchmark on your code and your environment.

The Mean tells us the average measured execution time.

Error describes uncertainty around that estimate.

StdDev shows variability.

Allocated tells us how much managed memory was allocated per operation.

That last column can be particularly valuable in ASP.NET Core.

A method that saves:

20 bytes

does not sound important.

But if it executes:

50 million times per hour

the economics change.

Performance depends on frequency.

Hot Paths Matter More Than Clever Code

A hot path is code that executes frequently enough, or consumes enough resources, to have a meaningful effect on application performance.

Imagine two methods.

Method A takes:

5 ms

and runs:

once per hour

Method B takes:

100 μs

and runs:

20,000 times per second

Which deserves attention?

Probably Method B.

Performance optimization should consider:

Cost per operation
        ×
Execution frequency

not merely which method looks slow in isolation.

Benchmark the Work That Matters

Good BenchmarkDotNet candidates include:

Serialization routines
Parsing
Mapping
Formatting
Compression
Hashing
Buffer manipulation
Algorithms
Collection operations
Custom middleware logic
Frequently executed transformations

Poor candidates include complete production workflows involving:

Remote databases
Internet APIs
Message brokers
DNS
Cloud storage
External authentication

Network and infrastructure variability can overwhelm the tiny code differences a microbenchmark is designed to measure.

If you want to understand the complete application, move up a level.

BenchmarkDotNet Does Not Tell You How Your API Scales

Suppose a method benchmark shows:

25 μs per operation

That does not mean your endpoint will process:

40,000 requests per second

An HTTP request may also involve:

Kestrel
Middleware
Authentication
Authorization
Model binding
Serialization
Database access
Network calls
Logging
Caching
Thread scheduling
Response transmission

A microbenchmark isolates one piece.

A load test measures the assembled system.

This distinction prevents a huge amount of misleading performance work.

What Load Testing Actually Measures

Suppose our application exposes:

GET /api/products/42

A load test can send requests from many simulated clients.

Perhaps:

10 users
     ↓
100 users
     ↓
500 users
     ↓
1,000 users

At each stage we observe:

Throughput
Latency
Error rate
CPU
Memory
GC
Thread Pool behavior
Database activity
Dependency latency
Queue depth

Now we are studying a system rather than a method.

Tools such as k6, JMeter, Gatling, Locust, Vegeta, and NBomber can all be used for this type of testing.

There is no requirement that every ASP.NET Core team use the same tool.

The testing model matters more than the logo on the tool.

Never Load-Test a Debug Build

This sounds obvious, but it causes surprisingly misleading results.

A Debug build is intended to support development and debugging, not to represent optimized production performance.

Likewise, Development configuration may enable additional diagnostics and logging that affect the results.

A setup like this:

Development laptop
Debug build
Verbose logging
Browser open
IDE debugger attached

is not a trustworthy production performance environment.

Prefer something closer to:

Release build
Production configuration
Production-like infrastructure
Representative database
Representative network
Representative resource limits

Perfect reproduction of production is not always possible.

But the closer the environment is, the more meaningful the conclusions become.

Throughput Is Only One Number

Suppose our load test reports:

2,500 requests/second

That sounds impressive.

But what if users wait eight seconds for responses?

Throughput measures how much work the system completes over time.

It does not tell us how long individual users wait.

We therefore need latency too.

Average Latency Can Hide Pain

Imagine ten requests:

50 ms
52 ms
48 ms
55 ms
51 ms
49 ms
53 ms
50 ms
51 ms
2,000 ms

Most requests are fast.

One is terrible.

The average rises, but it still hides the shape of the experience.

At scale, we usually care about percentiles.

For example:

p50 = 52 ms
p95 = 140 ms
p99 = 700 ms

p50 represents the median experience.

p95 means 95 percent of measured requests completed at or below that value.

p99 exposes behavior near the slow end of the distribution.

This matters because production problems often live in the tail.

Tail Latency Is Where Systems Reveal Themselves

Why might a small percentage of requests suddenly become slow?

Possible causes include:

Garbage collection
Database lock contention
Connection pool exhaustion
Thread Pool starvation
Slow downstream services
Cache misses
Disk activity
Queue buildup
Network delays
Retries

An average can smooth these events away.

Users experiencing the p99 do not care that the average looked healthy.

This is why serious load tests should report latency distributions rather than a single average.

Error Rate Belongs Beside Latency

Imagine two systems.

System A:

4,000 requests/sec
8% errors

System B:

3,500 requests/sec
0.02% errors

Which is faster?

That is the wrong question.

Performance and correctness cannot be separated.

A system can produce extraordinary throughput by failing quickly.

Your load-test dashboard should therefore put at least these together:

Throughput
Latency percentiles
Error rate

A performance result without errors is incomplete.

Capacity Is Not the Same as Maximum Throughput

Suppose we gradually increase load:

500 req/s
1,000 req/s
1,500 req/s
2,000 req/s
2,500 req/s
3,000 req/s

At first, latency remains stable.

Then something changes.

Perhaps:

Throughput plateaus
p95 latency rises
p99 explodes
Errors appear
Queues grow

We have reached a saturation region.

The application’s useful capacity is usually below the absolute point where it collapses.

Production systems need headroom.

If a service barely survives its expected peak traffic in a laboratory test, it does not have healthy production capacity.

Saturation Often Looks Like a Curve

Imagine:

Load      p95

500       60 ms
1000      65 ms
1500      72 ms
2000      90 ms
2500     180 ms
3000     850 ms
3500    3100 ms

Notice what happened.

Latency did not rise smoothly.

The system reached a point where additional work began waiting behind existing work.

That waiting creates more waiting.

This is exactly where the principles from

become important. Bounded queues, concurrency limits, and load shedding help prevent an overloaded application from continuing to accept more work than it can safely process.

Without those controls, a system can keep accepting requests after useful capacity has already been exhausted.

Load Testing Should Model Real Traffic

Suppose production traffic is:

60% GET /products
20% GET /search
10% POST /orders
5% GET /account
5% other

But our load test sends:

100% GET /health

The result tells us almost nothing about production capacity.

A realistic test should approximate:

Endpoint mix
Payload sizes
Authentication
Database behavior
Cache hit rates
User pacing
Concurrency
Data distribution
Downstream calls

This is particularly important with caching.

If every virtual user repeatedly requests:

/api/products/1

you may accidentally benchmark your cache rather than your application.

Warm Tests and Cold Tests Answer Different Questions

Consider a service that uses:

JIT compilation
Connection pools
Database caches
Application caches
DNS caches
TLS sessions

Its first few requests may behave differently from later requests.

That creates two legitimate questions.

Cold-start test:

How quickly can a new instance become useful?

Steady-state test:

How does the service perform after it has warmed up?

Do not mix them.

This becomes particularly important when evaluating Native AOT in ASP.NET Core: Faster Startup, Smaller Containers, and the Trade-Offs. Native AOT can substantially change startup characteristics, but a steady-state load test answers a different performance question.

The best way to determine whether Native AOT actually benefits your service is to benchmark both deployment models under conditions that match the problem you are trying to solve.

A Fast Endpoint Can Hide a Slow Dependency

Suppose:

API processing          8 ms
Database              140 ms
Payment service       250 ms
Serialization           5 ms

Developers may inspect the controller and spend hours optimizing those eight milliseconds.

But the controller is not the bottleneck.

The system is waiting on dependencies.

This is where distributed tracing becomes extremely useful.

A trace can show:

HTTP request
│
├── authentication       3 ms
├── application logic    5 ms
├── SQL query          142 ms
├── external API       247 ms
└── serialization        6 ms

Now optimization has direction.

Find the Bottleneck, Not the Most Interesting Code

Developers naturally gravitate toward code they control.

But the real bottleneck may be:

Database index
Network
DNS
Connection pool
Thread Pool
Lock
Disk
External API
Queue
Memory pressure
GC

The correct question is not:

What code can I make faster?

It is:

What resource currently limits the system?

That difference separates performance tuning from performance engineering.

CPU Saturation Tells One Story

Suppose load increases and CPU reaches:

95–100%

At the same time:

Throughput stops increasing
Latency rises

You probably have a CPU-bound bottleneck.

Now investigate:

Serialization
Compression
Encryption
Mapping
Algorithms
Excessive logging
Exception handling
Busy loops

A CPU profiler can identify hot code paths.

This may lead naturally back to BenchmarkDotNet.

The workflow becomes:

Load test
    ↓
CPU bottleneck found
    ↓
Profiler identifies hot method
    ↓
BenchmarkDotNet isolates method
    ↓
Optimization
    ↓
Benchmark again
    ↓
Load test again

That is much stronger than starting with a random microbenchmark.

Low CPU Does Not Mean the Application Has Capacity

Imagine latency is terrible but CPU sits at:

25%

That does not mean the application is healthy.

It may be waiting.

Perhaps threads are blocked on:

Database
Locks
Network calls
Synchronous I/O
Connection pools

This is a classic case where CPU alone gives the wrong impression.

The server is not busy computing.

It is busy waiting badly.

Thread Pool Starvation Can Look Mysterious

Imagine every request does this:

var result = SomeAsyncOperation().Result;

Under light traffic, everything may appear fine.

Under heavy concurrency, request threads become blocked waiting for asynchronous operations.

More requests arrive.

More threads become blocked.

The Thread Pool tries to compensate.

Latency rises dramatically.

This is exactly why a functional test cannot replace a load test.

The problem may become visible only when concurrency changes the system’s behavior.

Memory Metrics Tell Another Story

Suppose throughput remains healthy initially.

Over a longer test:

Memory rises
GC becomes more frequent
Pause time increases
Latency becomes unstable

Now the bottleneck may involve allocations or retention.

Watch metrics such as:

Allocation rate
Heap size
Gen 0 collections
Gen 1 collections
Gen 2 collections
LOH activity
GC pause time

This connects directly to Memory Management in ASP.NET Core: GC, Allocations, LOH, and Object Pooling, where we explored why allocation rate, retained memory, the Large Object Heap, and garbage collection can affect application latency and throughput.

A load test can reveal whether those memory-management decisions still behave well when hundreds or thousands of requests are competing for resources.

Database Bottlenecks Are Extremely Common

Consider:

var orders = await db.Orders
    .Where(x => x.CustomerId == customerId)
    .ToListAsync();

The C# looks harmless.

But perhaps:

CustomerId has no useful index

or the query returns:

80,000 rows

or an ORM pattern creates:

N + 1 queries

or the connection pool is exhausted.

The application can be slow while ASP.NET Core itself is doing almost nothing wrong.

Database telemetry must therefore be part of serious performance testing.

Test the Database With Representative Data

A query against:

1,000 rows

may behave beautifully.

Production contains:

300 million rows

That is a different system.

Performance tests need representative:

Table sizes
Indexes
Data distribution
Relationship density
Query patterns
Cache behavior

Synthetic data is fine if it reproduces the characteristics that matter.

Tiny development databases are not performance models.

Connection Pools Can Become Invisible Queues

Suppose the database connection pool can support a certain number of active connections.

Traffic increases.

Every request needs a database connection.

Eventually:

Request
   ↓
Wait for connection
   ↓
Execute query
   ↓
Return connection

Now the query itself may still take only:

20 ms

but the request spends:

400 ms

waiting to obtain a connection.

If you measure only SQL execution duration, you may conclude that the database is fast.

End-to-end tracing reveals the wait.

This is why bottleneck analysis requires multiple layers of telemetry.

Queues Need Measurement Too

A background system may process each job in:

50 ms

That sounds healthy.

But if jobs arrive faster than workers complete them:

Arrival rate     >     Processing rate

queue depth grows.

Users experience increasing delay even though individual jobs remain fast.

Measure:

Queue depth
Queue wait time
Processing time
Arrival rate
Completion rate

Queue latency is often more important than worker execution time.

Benchmark Allocations, Not Just Nanoseconds

Suppose two implementations produce:

A: 120 ns, 0 B allocated
B: 100 ns, 256 B allocated

Which is better?

There is not enough information.

If the operation runs rarely, B may be perfectly reasonable.

If it executes millions of times per second, the allocation difference could create additional GC pressure.

BenchmarkDotNet’s memory diagnostics are therefore extremely useful.

Performance decisions should consider:

Execution time
Allocations
Frequency
Concurrency
Complexity

not one number.

Small Benchmark Wins Can Be Noise

Suppose version B is:

1.7% faster

Should we rewrite production code?

Maybe not.

Ask:

Is the result statistically stable?

Is the method actually hot?

Does it affect end-to-end performance?

Does the new code become harder to maintain?

Will the difference survive real workloads?

A 2 percent microbenchmark improvement inside code responsible for 0.1 percent of request time is almost certainly irrelevant.

Optimization has an opportunity cost.

Performance Regressions Are Where Benchmarks Become Powerful

Benchmarking is not only for making code faster.

It is excellent for preventing code from becoming slower.

Suppose a critical parser currently processes data in:

40 μs

A future change accidentally increases it to:

90 μs

A benchmark suite can expose the regression before deployment.

Likewise, load tests can enforce service-level performance requirements.

For example:

p95 < 250 ms
error rate < 0.5%
throughput > 1,500 req/s

Now performance becomes something the team can test, rather than something users report after release.

Be Careful With Performance Gates in CI

There is a catch.

Shared CI runners can be noisy.

Other workloads may affect:

CPU
Scheduling
Memory
Disk
Network

A strict rule such as:

Fail build if benchmark is 2% slower

may produce false failures.

Use stable benchmarking environments for sensitive comparisons.

For ordinary CI, larger regression thresholds or historical trend analysis may be more practical.

Performance automation is valuable only when teams trust the signal.

Stress Testing Goes Beyond Expected Traffic

A load test asks whether the system handles expected conditions.

A stress test deliberately goes further.

Perhaps expected peak traffic is:

2,000 requests/sec

A stress test might push:

2,500
3,000
4,000
5,000

until the system degrades.

We are deliberately discovering the system’s limits.

But there is another important question.

What happens after the overload disappears?

What Happens After Overload?

Suppose traffic spikes to:

5× normal

Latency explodes.

Some requests fail.

Then traffic returns to normal.

What happens next?

Healthy system:

Load falls
   ↓
Queues drain
   ↓
Resources recover
   ↓
Latency normalizes

Unhealthy system:

Load falls
   ↓
Retry storm continues
   ↓
Queues remain enormous
   ↓
Connections remain exhausted
   ↓
Latency stays terrible

The second system survived the spike technically but failed operationally.

This connects directly to backpressure, retry design, and graceful recovery.

Soak Tests Find Problems Short Tests Miss

A five-minute test can show excellent results.

Run the same workload for:

8 hours

and perhaps:

Memory slowly climbs
Connections leak
Caches grow without bounds
Queue depth drifts upward
Temporary files accumulate
Latency gradually worsens

This is where soak testing becomes useful.

A soak test applies sustained workload for an extended period to expose degradation that takes time to develop.

It is particularly valuable for applications that are expected to run continuously.

Spike Tests Examine Sudden Change

Autoscaling systems often face sudden traffic.

Imagine:

500 req/s
     ↓
5,000 req/s
within 10 seconds

Can the system absorb the transition?

Does autoscaling react quickly enough?

Do existing instances collapse before new ones become ready?

Do connection pools become exhausted?

Does the database survive the surge?

Faster startup may improve scaling response, but only a realistic spike test can tell you whether it materially changes the system’s behavior.

The Load Generator Can Become the Bottleneck

This is an easy mistake.

Suppose your load-testing machine reaches:

100% CPU

while the server is at:

35% CPU

Requests stop increasing.

You might conclude:

The server maxes out at 10,000 requests per second.

But perhaps the client generating the traffic maxed out.

The test infrastructure must have enough capacity to overload the system under test.

Otherwise you benchmark your load generator.

For very large tests, traffic may need to originate from multiple machines or a managed load-testing service.

Network Placement Changes Results

A load generator on the same machine as the application measures something different from a user thousands of kilometers away.

Network latency can dominate short server operations.

So decide what you want to measure.

For application processing capacity:

Load generator close to service

may be appropriate.

For user experience:

Realistic geographic/network paths

matter.

Again, the test design follows the question.

Establish a Baseline Before Optimizing

Suppose we are about to change JSON serialization.

First record:

Throughput: 2,200 req/s
p50:         48 ms
p95:        130 ms
p99:        310 ms
Errors:     0.03%
CPU:        68%
Memory:     620 MB

These numbers are illustrative.

Then make one meaningful change.

Run the same test again.

Without a baseline, statements such as:

It feels faster.

are not performance engineering.

Change One Thing at a Time

Suppose we simultaneously:

Add caching
Change serializer
Upgrade .NET
Rewrite database query
Enable pooling
Increase connection limits

Performance improves by 35 percent.

What caused it?

We do not know.

Perhaps one change improved performance by 50 percent while another made it 15 percent worse.

Controlled experiments are powerful because they isolate causes.

A useful loop is:

Measure
   ↓
Form hypothesis
   ↓
Change one important variable
   ↓
Measure again
   ↓
Compare

Profile Before You Micro-Optimize

Suppose CPU profiling shows:

Database client          38%
JSON serialization       22%
Custom mapper            18%
Logging                  12%
Everything else          10%

Now you know where investigation is likely to pay off.

Benchmark the mapper.

Investigate serialization.

Check logging.

Do not spend three days optimizing something hidden inside the final 10 percent unless evidence points there.

Profiling gives microbenchmarks a purpose.

Instead of asking:

What can we benchmark?

we ask:

Which expensive path is worth isolating and improving?

Measure Production Too

Laboratory tests are essential.

Production telemetry is reality.

Production contains things your test environment may not reproduce:

Real customers
Unexpected payloads
Uneven traffic
Slow devices
Real geographic latency
Third-party failures
Organic database growth
Cache churn
Deployment events
Background jobs

Use production metrics to discover which scenarios deserve controlled testing.

The two systems reinforce each other:

Production reveals problem
        ↓
Controlled test reproduces it
        ↓
Profiler finds cause
        ↓
Benchmark validates optimization
        ↓
Load test validates system
        ↓
Production confirms improvement

That is a mature performance workflow.

A Practical Performance Investigation

Imagine an order API.

Users report occasional slow checkout.

Normal traffic looks healthy.

Under a realistic load test, we observe:

p50     120 ms
p95     850 ms
p99    2400 ms

CPU is only:

45%

So raw computation probably is not the main bottleneck.

Tracing shows many slow requests waiting for database connections.

Database metrics show the pool frequently reaching its limit.

Further investigation reveals that one endpoint performs several sequential queries per request.

We rewrite the data-access pattern to reduce unnecessary round trips.

Run the same load test again.

Suppose p95 falls substantially and connection waits disappear.

Now we have evidence.

Not:

The new code looks cleaner.

But:

We identified a resource bottleneck, changed the behavior causing it, and measured the improvement under the same workload.

That is performance engineering.

Do Not Optimize Away Readability for Tiny Wins

BenchmarkDotNet can tempt developers into competitions over nanoseconds.

Suppose:

Readable version      95 ns
Complex version       89 ns

The second implementation is 6 nanoseconds faster.

But the method runs:

20 times per request

and requests take:

200 ms

That optimization is almost certainly meaningless.

Performance work must still respect:

Maintainability
Correctness
Security
Reliability
Developer productivity

The fastest code is useless if nobody can safely change it.

Benchmarking Native AOT Properly

Suppose we want to determine whether Native AOT helps our service.

Do not benchmark only one number.

Compare:

JIT deployment
vs
Native AOT

across:

Publish size
Container size
Process startup
Time to readiness
Idle memory
Memory under load
Throughput
p50 latency
p95 latency
p99 latency
CPU
Build time

Native AOT may win dramatically in some categories and show little difference in others.

That is not a contradiction.

It is precisely why we benchmark.

Benchmarking High-Performance I/O

The same principle applies to our earlier I/O work.

Suppose we replace:

Read entire body
     ↓
Create string
     ↓
Parse

with:

Stream
   ↓
Parse incrementally

The most interesting result may not be raw execution speed.

Perhaps:

Latency changes slightly

but:

Peak memory drops dramatically
GC frequency decreases
Throughput under concurrency improves

A narrow benchmark could miss the actual benefit.

Choose measurements that match the optimization.

Benchmarking Memory Improvements

Suppose we introduce ArrayPool<T>.

A microbenchmark can measure:

Execution time
Allocated bytes
GC collections

But then run a realistic concurrent workload.

Why?

Because pooling is often valuable when repeated allocation pressure compounds across many requests.

An isolated benchmark tells us whether the operation allocates less.

A load test tells us whether that changes application behavior.

Both are useful.

Performance Is Multidimensional

A common mistake is searching for one universal score.

There isn’t one.

Performance includes:

Latency
Throughput
Startup
Memory
CPU
Allocation rate
Container size
Scalability
Recovery
Cost

Improving one can hurt another.

Compression may:

Reduce network bytes
Increase CPU

Caching may:

Reduce database latency
Increase memory

Pooling may:

Reduce allocations
Increase retained memory

Higher concurrency may:

Increase throughput
Increase contention

Performance decisions are trade-offs.

Define a Performance Budget

Instead of saying:

Make checkout fast.

Define:

p50 latency       < 150 ms
p95 latency       < 300 ms
p99 latency       < 750 ms
error rate        < 0.1%
memory            < 1 GB per instance
CPU               < 75% at target load
throughput        ≥ 2,000 req/s

Again, those are illustrative values.

Your business requirements should determine the real ones.

Now the team has a target.

Performance becomes testable.

A Useful Testing Pyramid for Performance

Think of performance testing at several levels:

              Production
                 ▲
             Stress/Soak
                 ▲
              Load Test
                 ▲
              Profiling
                 ▲
           Microbenchmark

At the bottom, BenchmarkDotNet gives precise measurements of small operations.

Moving upward introduces more of the real system.

By the time we reach production, realism is highest but experimental control is lowest.

No single layer replaces the others.

The Best Performance Workflow

A practical workflow looks like this.

First, define the requirement.

What must the system achieve?

Second, establish a baseline.

Record current performance.

Third, reproduce the workload.

Use representative traffic and data.

Fourth, observe the complete system.

Measure latency, throughput, errors, CPU, memory, GC, queues, databases, and dependencies.

Fifth, identify the limiting resource.

Do not guess.

Sixth, profile the relevant component.

Find the expensive path.

Seventh, isolate code where useful.

Use BenchmarkDotNet for focused comparisons.

Eighth, change one meaningful thing.

Keep the experiment understandable.

Ninth, rerun the same test.

Compare against the baseline.

Tenth, verify production behavior.

Make sure the laboratory improvement survives reality.

That workflow is far more valuable than memorizing a collection of “fast” coding tricks.

Common Benchmarking Mistakes

The most common mistakes are surprisingly consistent.

Benchmarking Debug builds. Measure optimized production-style builds.

Using a stopwatch for tiny operations. A proper benchmarking harness handles many sources of measurement error.

Looking only at averages. Tail latency matters.

Ignoring errors. Failed requests can make throughput look artificially impressive.

Testing unrealistic endpoints. Your health endpoint is not your production workload.

Using tiny databases. Data scale changes query behavior.

Ignoring warm-up. Cold and steady-state tests answer different questions.

Optimizing before profiling. Interesting code is not necessarily hot code.

Ignoring the load generator. Your client can become the bottleneck.

Changing several variables simultaneously. You lose causality.

Treating microbenchmarks as application benchmarks. A fast method does not guarantee a scalable service.

Ignoring production telemetry. Synthetic tests cannot reproduce every real-world condition.

Finding the Real Bottleneck Changes Everything

There is a powerful moment in performance work when the problem becomes specific.

Instead of:

Our API is slow.

we discover:

p99 latency rises after 1,800 requests per second because database connection waits increase sharply.

Or:

Throughput stops scaling because CPU is dominated by serialization.

Or:

Latency spikes correlate with Gen 2 garbage collections caused by repeated large-buffer allocation.

Or:

Requests pile up because synchronous blocking is starving the Thread Pool.

Those statements can be acted upon.

“Slow” cannot.

This is the real purpose of benchmarking.

Not generating impressive charts.

Not winning nanosecond competitions.

Turning vague performance complaints into measurable engineering problems.

Coming Next

In the next article, we’ll explore Contract-First API Development in ASP.NET Core: OpenAPI, Client Generation, and Schema Governance.

That may seem like a shift away from performance, but it continues the same broader theme: making production systems predictable.

Instead of allowing an API contract to emerge accidentally from implementation code, contract-first development treats the API schema as an explicit engineering artifact.

We’ll examine OpenAPI contracts, generated clients, breaking-change detection, schema governance, versioning decisions, and how teams keep API producers and consumers synchronized as systems grow.

Closing Thoughts

Performance optimization is seductive because changing code feels productive.

Measurement sometimes feels slower.

But measurement is what prevents us from spending days solving the wrong problem.

BenchmarkDotNet can tell us whether one small implementation is faster, more stable, or less allocation-heavy than another.

Profilers can tell us where the application spends CPU time.

Load tests can show how the complete system behaves under realistic concurrency.

Stress tests can reveal the point where the system stops coping.

Soak tests can expose problems that emerge slowly.

Production telemetry tells us whether any of those laboratory conclusions survive contact with real users.

Together, these tools give us a much more reliable picture of application performance than any single benchmark number.

The mature performance question is therefore not:

How can we make this code faster?

It is:

What is limiting the system right now, how do we prove it, and what measurement will prove that our change actually fixed it?

Once you can answer those questions, performance optimization stops being guesswork.

It becomes engineering.

Subscribe Now

Enjoying the series? Subscribe to ASP Today for practical ASP.NET Core tutorials, production architecture deep dives, and evidence-based .NET performance strategies. Join our Substack Chat to discuss BenchmarkDotNet, load testing, profiling, bottleneck analysis, Native AOT, memory optimization, and the techniques that help ASP.NET Core applications perform reliably under real-world traffic.