ASP.NET Core Background Services and Hangfire
Handle background operations in your ASP.NET Core applications with built-in services and Hangfire
Modern web applications often need to perform tasks that shouldn’t block user interactions, from sending emails and processing uploads to syncing data and generating reports. ASP.NET Core provides powerful built-in background services for simple scenarios, while Hangfire offers a robust solution for complex job scheduling and management.
This comprehensive guide explores both approaches, helping you choose the right tool for your long-running tasks and implement them effectively in production environments.
Understanding the Need for Background Processing
When users interact with your ASP.NET Core application, they expect immediate responses. Nobody wants to wait thirty seconds while your application processes a large file upload, sends dozens of notification emails, or generates a complex report. This is where background processing becomes essential. By offloading time-consuming operations to background tasks, you maintain a responsive user experience while ensuring critical work gets done reliably.
Background processing serves several crucial purposes in modern applications. It prevents request timeouts by moving lengthy operations outside the HTTP request lifecycle. It improves user experience by returning control to users immediately while processing continues behind the scenes. It enables better resource utilization by spreading work across time and available computing resources. Most importantly, it provides resilience through retry mechanisms and persistence that wouldn’t be possible within a single request.
Consider a typical e-commerce scenario where a customer places an order. The immediate response should confirm the order was received, but numerous background tasks follow: processing payment, updating inventory, sending confirmation emails, notifying warehouse systems, and generating invoices. Attempting all these operations synchronously would create an unacceptable user experience and increase the likelihood of failures disrupting the entire order process.
ASP.NET Core Background Services Fundamentals
ASP.NET Core introduced the IHostedService interface as part of its generic host infrastructure, providing a clean abstraction for running background tasks. This interface requires implementing two methods: StartAsync for initialization when the application starts, and StopAsync for cleanup when the application shuts down gracefully. The framework manages the lifecycle of these services, ensuring they start and stop appropriately with your application.
The BackgroundService base class simplifies implementation by providing a more convenient abstraction over IHostedService. Instead of managing the start and stop logic yourself, you override a single ExecuteAsync method where your background logic resides. This approach handles common patterns like cancellation tokens and exception handling, making it easier to write robust background services.
Here’s a practical example of a background service that processes queued email notifications:
public class EmailNotificationService : BackgroundService
{
private readonly IServiceProvider _serviceProvider;
private readonly ILogger<EmailNotificationService> _logger;
private readonly Channel<EmailMessage> _queue;
public EmailNotificationService(
IServiceProvider serviceProvider,
ILogger<EmailNotificationService> logger)
{
_serviceProvider = serviceProvider;
_logger = logger;
_queue = Channel.CreateUnbounded<EmailMessage>();
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await foreach (var email in _queue.Reader.ReadAllAsync(stoppingToken))
{
try
{
using var scope = _serviceProvider.CreateScope();
var emailSender = scope.ServiceProvider.GetRequiredService<IEmailSender>();
await emailSender.SendAsync(email);
_logger.LogInformation($"Email sent to {email.Recipient}");
}
catch (Exception ex)
{
_logger.LogError(ex, $"Failed to send email to {email.Recipient}");
// Implement retry logic or dead letter queue here
}
}
}
public async Task EnqueueEmailAsync(EmailMessage email)
{
await _queue.Writer.WriteAsync(email);
}
}This service demonstrates several important patterns. It uses dependency injection through IServiceProvider to access scoped services within the background context. It leverages System.Threading.Channels for thread-safe communication between the web request and background processing. It implements proper error handling and logging to ensure failures don’t crash the service. The separation of enqueueing and processing allows the web request to return immediately while email sending happens asynchronously.
Implementing Robust Background Services
Creating reliable background services requires careful attention to several critical aspects. Error handling becomes particularly important since exceptions in background services can terminate your entire application if not properly managed. Always wrap your processing logic in try-catch blocks and implement appropriate retry mechanisms for transient failures. Consider implementing exponential backoff for retries to avoid overwhelming failing services.
Resource management in background services demands special consideration. Since these services run for the entire lifetime of your application, memory leaks or resource exhaustion can accumulate over time. Always dispose of resources properly, create new dependency injection scopes for scoped services, and monitor memory usage in production. When working with databases, ensure connections are properly released after each operation to avoid connection pool exhaustion.
Graceful shutdown handling ensures your background services stop cleanly when the application shuts down. The CancellationToken passed to ExecuteAsync signals when shutdown is requested. Honor this token by checking it regularly in long-running loops and passing it to async operations. Implement timeout logic in StopAsync to prevent hanging during shutdown if cleanup takes too long.
Here’s an enhanced version that demonstrates these principles:
public class ResilientBackgroundService : BackgroundService
{
private readonly IServiceProvider _serviceProvider;
private readonly ILogger<ResilientBackgroundService> _logger;
private readonly SemaphoreSlim _semaphore;
public ResilientBackgroundService(
IServiceProvider serviceProvider,
ILogger<ResilientBackgroundService> logger)
{
_serviceProvider = serviceProvider;
_logger = logger;
_semaphore = new SemaphoreSlim(5); // Limit concurrent operations
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await _semaphore.WaitAsync(stoppingToken);
try
{
await ProcessNextItemAsync(stoppingToken);
}
catch (OperationCanceledException)
{
// Expected during shutdown
break;
}
catch (Exception ex)
{
_logger.LogError(ex, "Unhandled exception in background service");
await Task.Delay(TimeSpan.FromSeconds(5), stoppingToken);
}
finally
{
_semaphore.Release();
}
}
}
private async Task ProcessNextItemAsync(CancellationToken cancellationToken)
{
using var scope = _serviceProvider.CreateScope();
var processor = scope.ServiceProvider.GetRequiredService<IDataProcessor>();
var item = await processor.GetNextItemAsync(cancellationToken);
if (item != null)
{
await ProcessWithRetryAsync(item, processor, cancellationToken);
}
else
{
await Task.Delay(TimeSpan.FromSeconds(10), cancellationToken);
}
}
private async Task ProcessWithRetryAsync(
DataItem item,
IDataProcessor processor,
CancellationToken cancellationToken)
{
var retryCount = 0;
var maxRetries = 3;
while (retryCount < maxRetries)
{
try
{
await processor.ProcessAsync(item, cancellationToken);
return;
}
catch (Exception ex) when (retryCount < maxRetries - 1)
{
retryCount++;
var delay = TimeSpan.FromSeconds(Math.Pow(2, retryCount));
_logger.LogWarning(ex, $"Processing failed, retry {retryCount} after {delay}");
await Task.Delay(delay, cancellationToken);
}
}
}
}Introduction to Hangfire
While ASP.NET Core’s built-in background services work well for simple scenarios, complex applications often need more sophisticated job management capabilities. Hangfire provides a comprehensive solution for background job processing with features like persistence, retries, scheduling, and a dashboard for monitoring. It transforms background job processing from a coding challenge into a configuration and monitoring task.
Hangfire’s architecture consists of three main components: storage, server, and client. The storage component persists job definitions and state, supporting various backends including SQL Server, PostgreSQL, and Redis. The server component pulls jobs from storage and executes them using worker threads. The client component enqueues new jobs and schedules recurring tasks. This separation allows you to scale different components independently based on your application’s needs.
Setting up Hangfire in your ASP.NET Core application requires just a few configuration steps. First, install the necessary NuGet packages for Hangfire.Core and your chosen storage provider. Then configure Hangfire in your Program.cs file:
var builder = WebApplication.CreateBuilder(args);
// Add Hangfire services
builder.Services.AddHangfire(configuration => configuration
.SetDataCompatibilityLevel(CompatibilityLevel.Version_180)
.UseSimpleAssemblyNameTypeSerializer()
.UseRecommendedSerializerSettings()
.UseSqlServerStorage(builder.Configuration.GetConnectionString("HangfireConnection")));
// Add the Hangfire server
builder.Services.AddHangfireServer(options =>
{
options.WorkerCount = Environment.ProcessorCount * 2;
options.Queues = new[] { "critical", "default", "low" };
});
var app = builder.Build();
// Add Hangfire Dashboard (protected in production)
app.UseHangfireDashboard("/hangfire", new DashboardOptions
{
Authorization = new[] { new HangfireAuthorizationFilter() }
});
app.Run();This configuration establishes Hangfire with SQL Server storage, configures the background job server with multiple queues for priority handling, and adds the dashboard for monitoring job execution. The dashboard provides real-time insights into job processing, including succeeded, failed, and processing jobs, along with detailed execution history.
Hangfire Job Types and Patterns
Hangfire supports several job types that cover most background processing scenarios. Fire-and-forget jobs execute once as soon as possible after creation, perfect for tasks like sending emails or processing uploads. Delayed jobs execute once after a specified delay, useful for scheduling follow-up actions. Recurring jobs execute on a schedule defined using CRON expressions, ideal for periodic maintenance tasks. Continuations execute after another job completes, enabling complex workflows.
Fire-and-forget jobs represent the most common pattern. You enqueue them from your controller actions or service methods, and Hangfire ensures they execute at least once:
public class OrderController : ControllerBase
{
private readonly IBackgroundJobClient _backgroundJobs;
private readonly IOrderService _orderService;
public OrderController(
IBackgroundJobClient backgroundJobs,
IOrderService orderService)
{
_backgroundJobs = backgroundJobs;
_orderService = orderService;
}
[HttpPost]
public async Task<IActionResult> CreateOrder([FromBody] OrderRequest request)
{
var order = await _orderService.CreateOrderAsync(request);
// Enqueue background jobs
_backgroundJobs.Enqueue<IOrderProcessor>(x => x.ProcessPaymentAsync(order.Id));
_backgroundJobs.Enqueue<IInventoryService>(x => x.UpdateInventoryAsync(order.Id));
_backgroundJobs.Enqueue<INotificationService>(x => x.SendOrderConfirmationAsync(order.Id));
return Ok(new { OrderId = order.Id, Status = "Processing" });
}
}Recurring jobs enable scheduled task execution without external schedulers or cron jobs. Hangfire manages the scheduling complexity, ensuring jobs run according to their CRON expressions even across application restarts:
public class RecurringJobConfiguration
{
public static void ConfigureRecurringJobs()
{
// Daily report generation at 2 AM
RecurringJob.AddOrUpdate<IReportService>(
"daily-sales-report",
x => x.GenerateDailySalesReportAsync(),
"0 2 * * *",
TimeZoneInfo.Local);
// Hourly cache refresh
RecurringJob.AddOrUpdate<ICacheService>(
"cache-refresh",
x => x.RefreshCacheAsync(),
"0 * * * *");
// Weekly database maintenance on Sunday at 3 AM
RecurringJob.AddOrUpdate<IMaintenanceService>(
"weekly-maintenance",
x => x.PerformDatabaseMaintenanceAsync(),
"0 3 * * 0");
}
}Advanced Hangfire Features
Hangfire’s advanced features enable sophisticated job processing scenarios that would be complex to implement from scratch. Job filters provide AOP-style interception points for cross-cutting concerns like logging, performance monitoring, and transaction management. You can create custom filters by implementing IClientFilter, IServerFilter, IElectStateFilter, or IApplyStateFilter interfaces.
Here’s an example of a custom filter that adds detailed logging and performance tracking:
public class JobPerformanceFilter : JobFilterAttribute, IServerFilter
{
private static readonly ILogger Logger = LogManager.GetCurrentClassLogger();
public void OnPerforming(PerformingContext context)
{
Logger.Info($"Starting job {context.BackgroundJob.Id} - {context.BackgroundJob.Job.Method.Name}");
context.SetJobParameter("startTime", DateTimeOffset.UtcNow);
}
public void OnPerformed(PerformedContext context)
{
var startTime = context.GetJobParameter<DateTimeOffset>("startTime");
var duration = DateTimeOffset.UtcNow - startTime;
Logger.Info($"Completed job {context.BackgroundJob.Id} in {duration.TotalMilliseconds}ms");
if (context.Exception != null)
{
Logger.Error(context.Exception, $"Job {context.BackgroundJob.Id} failed");
}
}
}Batch operations allow you to create and monitor groups of related jobs as a single unit. This proves invaluable for complex workflows where you need to track the overall progress of multiple operations:
public class DataImportService
{
private readonly IBackgroundJobClient _backgroundJobs;
public async Task ImportLargeDatasetAsync(string filePath)
{
var chunks = await SplitFileIntoChunksAsync(filePath);
var batchId = BatchJob.StartNew(x =>
{
foreach (var chunk in chunks)
{
x.Enqueue<IDataProcessor>(p => p.ProcessChunkAsync(chunk));
}
});
// Create a continuation job that runs after all chunks are processed
BatchJob.ContinueWith(batchId, x =>
{
x.Enqueue<IDataProcessor>(p => p.MergeProcessedChunksAsync());
});
}
}Job continuations enable workflow orchestration by chaining jobs together. Each job in the chain executes only after its predecessor completes successfully. This pattern works well for multi-step processes where each step depends on the previous one’s output.
Choosing Between Background Services and Hangfire
The decision between using ASP.NET Core’s built-in background services and Hangfire depends on your specific requirements and constraints. Built-in background services excel in scenarios requiring minimal overhead, real-time processing, or tight integration with your application’s lifecycle. They’re ideal for simple recurring tasks, in-memory queue processing, and situations where you want to avoid external dependencies.
Hangfire becomes the better choice when you need persistent job storage, complex scheduling, job retry mechanisms, or visibility into job execution through its dashboard. It handles scenarios where jobs must survive application restarts, require sophisticated error handling and retry policies, or need to be distributed across multiple servers. The dashboard alone often justifies Hangfire adoption in production environments where operations teams need visibility into background job processing.
Consider a hybrid approach for complex applications. Use built-in background services for high-frequency, low-latency operations like real-time notifications or cache updates. Deploy Hangfire for scheduled jobs, batch processing, and operations requiring persistence and monitoring. This combination leverages the strengths of both approaches while minimizing overhead.
Performance considerations also influence this decision. Built-in background services have virtually no overhead beyond your actual processing logic. Hangfire introduces some overhead through job serialization, storage operations, and polling mechanisms. For high-throughput scenarios processing thousands of jobs per second, built-in services with in-memory queues might perform better. For reliability-critical scenarios where losing jobs is unacceptable, Hangfire’s persistence makes it the clear choice.
Production Considerations and Best Practices
Deploying background processing to production requires careful attention to monitoring, scaling, and reliability concerns. Comprehensive logging becomes crucial since background jobs execute outside the normal request/response cycle. Implement structured logging that includes job identifiers, execution times, and meaningful context to troubleshoot issues effectively.
Monitoring background job health requires both application-level and infrastructure-level metrics. Track job execution times, failure rates, and queue depths at the application level. Monitor CPU usage, memory consumption, and thread pool utilization at the infrastructure level. Set up alerts for anomalies like increasing queue depths or rising failure rates that might indicate problems before they impact users.
Database considerations become critical when using Hangfire with SQL storage. The Hangfire schema can grow significantly in high-volume environments, so implement retention policies to clean up old job data. Index optimization might be necessary for large job tables. Consider using a separate database for Hangfire to isolate its impact on your application database. For extreme scale, evaluate Redis as an alternative storage backend that handles high throughput better than relational databases.
Here’s a production-ready configuration example that addresses these concerns:
public class ProductionHangfireConfiguration
{
public static void Configure(IServiceCollection services, IConfiguration configuration)
{
services.AddHangfire(config => config
.SetDataCompatibilityLevel(CompatibilityLevel.Version_180)
.UseSimpleAssemblyNameTypeSerializer()
.UseRecommendedSerializerSettings()
.UseSqlServerStorage(configuration.GetConnectionString("HangfireConnection"),
new SqlServerStorageOptions
{
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
QueuePollInterval = TimeSpan.Zero,
UseRecommendedIsolationLevel = true,
DisableGlobalLocks = true,
SchemaName = "HangfireSchema"
})
.UseFilter(new AutomaticRetryAttribute { Attempts = 3 })
.UseFilter(new JobPerformanceFilter())
.UseFilter(new PreserveOriginalQueueAttribute()));
services.AddHangfireServer(options =>
{
options.ServerName = $"{Environment.MachineName}:{Guid.NewGuid()}";
options.WorkerCount = configuration.GetValue<int>("Hangfire:WorkerCount", 20);
options.Queues = configuration.GetSection("Hangfire:Queues").Get<string[]>()
?? new[] { "critical", "default", "low" };
options.ShutdownTimeout = TimeSpan.FromMinutes(1);
options.StopTimeout = TimeSpan.FromSeconds(30);
});
}
}Scaling strategies differ between background services and Hangfire. Built-in background services scale vertically within a single application instance by adjusting thread pools and concurrent operation limits. Horizontal scaling requires implementing distributed coordination through external services like Redis or a message queue. Hangfire naturally supports horizontal scaling by running multiple server instances against the same storage backend. It automatically distributes jobs across available workers, though you need to ensure your jobs are idempotent since the same job might execute multiple times in failure scenarios.
Security and Error Handling
Security considerations for background processing often get overlooked but remain critical for production applications. Background jobs typically execute with elevated privileges since they run within your application context. Validate and sanitize all job parameters to prevent injection attacks. Implement proper authentication and authorization for the Hangfire dashboard to prevent unauthorized access to sensitive job data.
Error handling strategies must account for the asynchronous nature of background processing. Unlike synchronous operations where errors immediately return to the caller, background job failures might go unnoticed without proper monitoring. Implement comprehensive error logging that captures full exception details including stack traces. Consider implementing a dead letter queue pattern for jobs that fail repeatedly after maximum retry attempts.
Here’s a comprehensive error handling implementation:
public class ErrorHandlingJobService
{
private readonly ILogger<ErrorHandlingJobService> _logger;
private readonly IServiceProvider _serviceProvider;
private readonly IMetrics _metrics;
public async Task ProcessJobWithErrorHandling(JobContext context)
{
var jobId = context.JobId;
var attemptCount = context.RetryAttempt;
try
{
_metrics.IncrementJobAttempts(jobId);
await ExecuteJobLogicAsync(context);
_metrics.RecordJobSuccess(jobId);
}
catch (TransientException ex)
{
_logger.LogWarning(ex, $"Transient error in job {jobId}, attempt {attemptCount}");
_metrics.RecordJobRetry(jobId);
if (attemptCount >= 3)
{
await MoveToDeadLetterQueueAsync(context, ex);
}
throw; // Let Hangfire handle retry
}
catch (PermanentException ex)
{
_logger.LogError(ex, $"Permanent error in job {jobId}");
_metrics.RecordJobFailure(jobId);
await MoveToDeadLetterQueueAsync(context, ex);
// Don't rethrow - job won't be retried
}
catch (Exception ex)
{
_logger.LogCritical(ex, $"Unexpected error in job {jobId}");
_metrics.RecordJobCriticalFailure(jobId);
await NotifyOperationsTeamAsync(jobId, ex);
throw;
}
}
private async Task MoveToDeadLetterQueueAsync(JobContext context, Exception ex)
{
using var scope = _serviceProvider.CreateScope();
var deadLetterService = scope.ServiceProvider.GetRequiredService<IDeadLetterService>();
await deadLetterService.StoreFailedJobAsync(new FailedJob
{
JobId = context.JobId,
JobType = context.JobType,
Parameters = context.Parameters,
FailureReason = ex.Message,
StackTrace = ex.StackTrace,
FailedAt = DateTimeOffset.UtcNow,
RetryCount = context.RetryAttempt
});
}
}Testing Background Services
Testing background services presents unique challenges compared to testing synchronous code. The asynchronous nature and long-running characteristics require special testing strategies. Unit tests should focus on the business logic within your background services, mocking external dependencies and verifying the correct behavior under various conditions.
Integration testing becomes particularly important for background services since they often interact with multiple system components. Use the ASP.NET Core TestServer to host your background services in tests, allowing you to verify their behavior in a realistic environment:
public class BackgroundServiceIntegrationTests
{
[Fact]
public async Task EmailService_ProcessesQueuedEmails()
{
// Arrange
var builder = new HostBuilder()
.ConfigureServices(services =>
{
services.AddSingleton<IEmailSender, MockEmailSender>();
services.AddHostedService<EmailNotificationService>();
services.AddSingleton<EmailNotificationService>();
});
using var host = await builder.StartAsync();
var emailService = host.Services.GetRequiredService<EmailNotificationService>();
var mockSender = host.Services.GetRequiredService<IEmailSender>() as MockEmailSender;
// Act
await emailService.EnqueueEmailAsync(new EmailMessage
{
Recipient = "[email protected]",
Subject = "Test",
Body = "Test email"
});
// Assert
await Task.Delay(TimeSpan.FromSeconds(1)); // Allow processing time
Assert.Single(mockSender.SentEmails);
Assert.Equal("[email protected]", mockSender.SentEmails.First().Recipient);
}
}For Hangfire testing, consider using the Hangfire.MemoryStorage package during tests to avoid database dependencies. This allows you to test job enqueueing and processing logic without the overhead of database operations:
public class HangfireJobTests
{
[Fact]
public void OrderProcessor_EnqueuesCorrectJobs()
{
// Arrange
var storage = new MemoryStorage();
GlobalConfiguration.Configuration.UseStorage(storage);
var client = new BackgroundJobClient();
var orderProcessor = new OrderProcessor(client);
// Act
orderProcessor.ProcessNewOrder(123);
// Assert
var monitoring = storage.GetMonitoringApi();
var enqueuedJobs = monitoring.EnqueuedJobs("default", 0, 10);
Assert.Equal(3, enqueuedJobs.Count); // Payment, inventory, notification
}
}Performance Optimization
Optimizing background service performance requires understanding the specific bottlenecks in your processing pipeline. Common performance issues include thread starvation, database connection exhaustion, memory leaks, and inefficient job serialization. Profile your background services under load to identify these bottlenecks before they impact production.
Thread pool configuration significantly impacts background service performance. The default thread pool settings might not be optimal for background-heavy workloads. Consider adjusting the minimum thread count to avoid thread starvation during burst processing:
public class Program
{
public static void Main(string[] args)
{
// Configure thread pool for background processing
ThreadPool.GetMinThreads(out var workerThreads, out var ioThreads);
ThreadPool.SetMinThreads(Math.Max(workerThreads, 100), Math.Max(ioThreads, 100));
CreateHostBuilder(args).Build().Run();
}
}Batching operations can dramatically improve throughput for high-volume scenarios. Instead of processing items individually, accumulate them and process in batches. This reduces overhead from database round trips, API calls, and other I/O operations:
public class BatchProcessingService : BackgroundService
{
private readonly Channel<WorkItem> _queue;
private readonly List<WorkItem> _batch = new();
private readonly SemaphoreSlim _batchLock = new(1);
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
var batchTimer = new Timer(
callback: async _ => await ProcessBatchAsync(),
state: null,
dueTime: TimeSpan.FromSeconds(5),
period: TimeSpan.FromSeconds(5));
await foreach (var item in _queue.Reader.ReadAllAsync(stoppingToken))
{
await _batchLock.WaitAsync(stoppingToken);
try
{
_batch.Add(item);
if (_batch.Count >= 100) // Process when batch is full
{
await ProcessBatchAsync();
}
}
finally
{
_batchLock.Release();
}
}
}
private async Task ProcessBatchAsync()
{
await _batchLock.WaitAsync();
try
{
if (_batch.Count == 0) return;
var itemsToProcess = _batch.ToList();
_batch.Clear();
// Process entire batch with single database operation
await BulkProcessItemsAsync(itemsToProcess);
}
finally
{
_batchLock.Release();
}
}
}Memory optimization becomes crucial for long-running services. Avoid holding references to large objects longer than necessary. Use object pooling for frequently allocated objects to reduce garbage collection pressure. Monitor memory usage patterns and investigate any steady growth that might indicate memory leaks.
Real-World Implementation Patterns
Implementing background processing in production applications often requires combining multiple patterns to achieve reliability and performance goals. The outbox pattern ensures reliable message publishing by storing events in the database within the same transaction as your business logic, then publishing them through a background service:
public class OutboxProcessor : BackgroundService
{
private readonly IServiceProvider _serviceProvider;
private readonly IMessagePublisher _publisher;
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
using var scope = _serviceProvider.CreateScope();
var dbContext = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();
var pendingMessages = await dbContext.OutboxMessages
.Where(m => m.ProcessedAt == null)
.OrderBy(m => m.CreatedAt)
.Take(100)
.ToListAsync(stoppingToken);
foreach (var message in pendingMessages)
{
try
{
await _publisher.PublishAsync(message.Payload);
message.ProcessedAt = DateTimeOffset.UtcNow;
}
catch (Exception ex)
{
message.FailureCount++;
message.LastError = ex.Message;
}
}
await dbContext.SaveChangesAsync(stoppingToken);
await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
}
}
}The saga pattern coordinates complex workflows spanning multiple services or long time periods. Each step in the saga can compensate for failures in subsequent steps, ensuring consistency even when operations fail:
public class OrderSaga
{
private readonly IBackgroundJobClient _jobClient;
public void StartOrderProcessing(int orderId)
{
var paymentJobId = _jobClient.Enqueue<IPaymentService>(
s => s.ProcessPaymentAsync(orderId));
var inventoryJobId = _jobClient.ContinueJobWith<IInventoryService>(
paymentJobId,
s => s.ReserveInventoryAsync(orderId));
var shippingJobId = _jobClient.ContinueJobWith<IShippingService>(
inventoryJobId,
s => s.CreateShippingLabelAsync(orderId));
// Compensation job if shipping fails
_jobClient.ContinueJobWith<ICompensationService>(
shippingJobId,
s => s.CompensateFailedOrderAsync(orderId),
JobContinuationOptions.OnlyOnFailedState);
}
}Circuit breaker patterns prevent cascading failures when external services become unavailable. By tracking failure rates and temporarily stopping attempts to failing services, you protect your application from wasting resources on doomed operations:
public class CircuitBreakerService
{
private readonly SemaphoreSlim _semaphore = new(1);
private int _failureCount;
private DateTimeOffset _lastFailureTime;
private CircuitState _state = CircuitState.Closed;
public async Task<T> ExecuteAsync<T>(Func<Task<T>> operation)
{
if (_state == CircuitState.Open)
{
if (DateTimeOffset.UtcNow - _lastFailureTime > TimeSpan.FromMinutes(1))
{
_state = CircuitState.HalfOpen;
}
else
{
throw new CircuitOpenException("Circuit breaker is open");
}
}
try
{
var result = await operation();
if (_state == CircuitState.HalfOpen)
{
await ResetCircuitAsync();
}
return result;
}
catch (Exception)
{
await RecordFailureAsync();
throw;
}
}
private async Task RecordFailureAsync()
{
await _semaphore.WaitAsync();
try
{
_failureCount++;
_lastFailureTime = DateTimeOffset.UtcNow;
if (_failureCount >= 5)
{
_state = CircuitState.Open;
}
}
finally
{
_semaphore.Release();
}
}
private async Task ResetCircuitAsync()
{
await _semaphore.WaitAsync();
try
{
_failureCount = 0;
_state = CircuitState.Closed;
}
finally
{
_semaphore.Release();
}
}
}Join The Community
Ready to implement robust background processing in your ASP.NET Core applications? Subscribe to ASP Today for weekly deep dives into advanced ASP.NET Core topics, and join our Substack Chat community where developers share experiences and solutions for building enterprise-grade applications.