|
Scheduling Background Jobs With Quartz in .NET (advanced concepts)
Read on: my website / Read time: 6 minutes
|
|
|
|
The .NET Weekly is brought to you by:
It's been a big year for API collaborations! The Postman VS Code extension helped us integrate our workflows, new templates got us started when we were stuck, and the Postman Vault kept our data secure. Postman's December drop has all the details on these and more.
|
|
|
|
Most ASP.NET Core applications need to handle background processing - from sending reminder emails to running cleanup tasks. While there are many ways to implement background jobs, Quartz.NET stands out with its robust scheduling capabilities, persistence options, and production-ready features.
In this article, we'll look at:
- Setting up Quartz.NET with ASP.NET Core and proper observability
- Implementing both on-demand and recurring jobs
- Configuring persistent storage with PostgreSQL
- Handling job data and monitoring execution
Let's start with the basic setup and build our way up to a production-ready configuration.
|
|
|
Setting Up Quartz With ASP.NET Core
First, let's set up Quartz with proper instrumentation.
We'll need to install some NuGet packages:
Install-Package Quartz.Extensions.Hosting
Install-Package Quartz.Serialization.Json
Install-Package OpenTelemetry.Instrumentation.Quartz
Next, we'll configure the Quartz services and OpenTelemetry instrumentation and start the scheduler:
builder.Services.AddQuartz();
builder.Services.AddQuartzHostedService(options =>
{
options.WaitForJobsToComplete = true;
});
builder.Services.AddOpenTelemetry()
.WithTracing(tracing =>
{
tracing
.AddHttpClientInstrumentation()
.AddAspNetCoreInstrumentation()
.AddQuartzInstrumentation();
})
.UseOtlpExporter();
This is all we need at the start.
|
|
|
Defining and Scheduling Jobs
To define a background job, you have to implement the IJob interface. All job implementations run as scoped services, so you can inject dependencies as needed. Quartz allows you to pass data to a job using the JobDataMap dictionary. It's recommended to only use primitive types for job data to avoid serialization issues.
When executing the job, there are a few ways to fetch job data:
-
JobDataMap - a dictionary of key-value pairs
JobExecutionContext.JobDetail.JobDataMap - job-specific dataJobExecutionContext.Trigger.TriggerDataMap - trigger-specific data
MergedJobDataMap - combines job data with trigger data
It's a best practice to use MergedJobDataMap to retrieve job data.
public class EmailReminderJob(ILogger<EmailReminderJob> logger, IEmailService emailService) : IJob
{
public const string Name = nameof(EmailReminderJob);
public async Task Execute(IJobExecutionContext context)
{
var data = context.MergedJobDataMap;
string? userId = data.GetString("userId");
string? message = data.GetString("message");
try
{
await emailService.SendReminderAsync(userId, message);
logger.LogInformation("Sent reminder to user {UserId}: {Message}", userId, message);
}
catch (Exception ex)
{
logger.LogError(ex, "Failed to send reminder to user {UserId}", userId);
throw;
}
}
}
One thing to note: JobDataMap isn't strongly typed. This is a limitation we have to live with, but we can mitigate it by:
- Using constants for key names
- Validating data early in the
Execute method
- Creating wrapper services for job scheduling
Now, let's discuss scheduling jobs.
Here's how to schedule one-time reminders:
public record ScheduleReminderRequest(
string UserId,
string Message,
DateTime ScheduleTime
);
app.MapPost("/api/reminders/schedule", async (
ISchedulerFactory schedulerFactory,
ScheduleReminderRequest request) =>
{
var scheduler = await schedulerFactory.GetScheduler();
var jobData = new JobDataMap
{
{ "userId", request.UserId },
{ "message", request.Message }
};
var job = JobBuilder.Create<EmailReminderJob>()
.WithIdentity($"reminder-{Guid.NewGuid()}", "email-reminders")
.SetJobData(jobData)
.Build();
var trigger = TriggerBuilder.Create()
.WithIdentity($"trigger-{Guid.NewGuid()}", "email-reminders")
.StartAt(request.ScheduleTime)
.Build();
await scheduler.ScheduleJob(job, trigger);
return Results.Ok(new { scheduled = true, scheduledTime = request.ScheduleTime });
})
.WithName("ScheduleReminder")
.WithOpenApi();
The endpoint schedules one-time email reminders using Quartz. It creates a job with user data, sets up a trigger for the specified time, and schedules them together. The EmailReminderJob receives a unique identity in the email-reminders group.
Here's a sample request you can use to test this out:
POST /api/reminders/schedule
{
"userId": "user123",
"message": "Important meeting!",
"scheduleTime": "2024-12-17T15:00:00"
}
|
|
|
Scheduling Recurring Jobs
For recurring background jobs, you can use cron schedules:
public record RecurringReminderRequest(
string UserId,
string Message,
string CronExpression
);
app.MapPost("/api/reminders/schedule/recurring", async (
ISchedulerFactory schedulerFactory,
RecurringReminderRequest request) =>
{
var scheduler = await schedulerFactory.GetScheduler();
var jobData = new JobDataMap
{
{ "userId", request.UserId },
{ "message", request.Message }
};
var job = JobBuilder.Create<EmailReminderJob>()
.WithIdentity($"recurring-{Guid.NewGuid()}", "recurring-reminders")
.SetJobData(jobData)
.Build();
var trigger = TriggerBuilder.Create()
.WithIdentity($"recurring-trigger-{Guid.NewGuid()}", "recurring-reminders")
.WithCronSchedule(request.CronExpression)
.Build();
await scheduler.ScheduleJob(job, trigger);
return Results.Ok(new { scheduled = true, cronExpression = request.CronExpression });
})
.WithName("ScheduleRecurringReminder")
.WithOpenApi();
Cron triggers are more powerful than simple triggers. They allow you to define complex schedules like "every weekday at 10 AM" or "every 15 minutes". Quartz supports cron expressions with seconds, minutes, hours, days, months, and years.
Here's a sample request if you want to test this:
POST /api/reminders/schedule/recurring
{
"userId": "user123",
"message": "Daily standup",
"cronExpression": "0 0 10 ? * MON-FRI"
}
|
|
|
Job Persistence Setup
By default, Quartz uses in-memory storage, which means your jobs are lost when the application restarts. For production environments, you'll want to use a persistent store. Quartz supports several database providers, including SQL Server, PostgreSQL, MySQL, and Oracle.
Let's look at how to set up persistent storage with proper schema isolation:
builder.Services.AddQuartz(options =>
{
options.AddJob<EmailReminderJob>(c => c
.StoreDurably()
.WithIdentity(EmailReminderJob.Name));
options.UsePersistentStore(persistenceOptions =>
{
persistenceOptions.UsePostgres(cfg =>
{
cfg.ConnectionString = connectionString;
cfg.TablePrefix = "scheduler.qrtz_";
},
dataSourceName: "reminders");
persistenceOptions.UseNewtonsoftJsonSerializer();
persistenceOptions.UseProperties = true;
});
});
A few important things to note here:
- The
TablePrefix setting helps organize Quartz tables in your database - in this case, placing them in a dedicated scheduler schema
- You'll need to run the appropriate database scripts to create these tables
- Each database provider has its own setup scripts - check the Quartz documentation for your chosen provider
Durable Jobs
Notice how we're configuring the EmailReminderJob with StoreDurably? This is a powerful pattern that lets you define your jobs once and reuse them with different triggers. Here's how to schedule a stored job:
public async Task ScheduleReminder(string userId, string message, DateTime scheduledTime)
{
var scheduler = await _schedulerFactory.GetScheduler();
var jobKey = new JobKey(EmailReminderJob.Name);
var trigger = TriggerBuilder.Create()
.ForJob(jobKey)
.WithIdentity($"trigger-{Guid.NewGuid()}")
.UsingJobData("userId", userId)
.UsingJobData("message", message)
.StartAt(scheduledTime)
.Build();
await scheduler.ScheduleJob(trigger);
}
This approach has several benefits:
- Job definitions are centralized in your startup configuration
- You can't accidentally schedule a job that hasn't been properly configured
- Job configurations are consistent across all schedules
|
|
|
Summary
Getting Quartz set up properly in .NET involves more than just adding the NuGet package.
Pay attention to:
- Proper job definition and data handling with
JobDataMap
- Setting up both one-time and recurring job schedules
- Configuring persistent storage with proper schema isolation
- Using durable jobs to maintain consistent job definitions
Each of these elements contributes to a reliable background processing system that can grow with your application's needs. A good example of using background jobs is when you want to build asynchronous APIs.
Good luck out there, and I'll see you next week.
|
|
|
Whenever you're ready, there are 3 ways I can help you:
|
|
Pragmatic Clean Architecture: This comprehensive course will teach you the system I use to ship production-ready applications using Clean Architecture. Learn how to apply the best practices of modern software architecture. Join 3,600+ engineers |
The REST APIs course will launch next month!
REST APIs in ASP .NET Core: You will learn how to build production-ready REST APIs using the latest ASP .NET Core features and best practices. Join the waitlist to get a special launch discount.
|
|
|
You received this email because you subscribed to our list. You can unsubscribe at any time.
Update your profile | Dragiše Cvetkovića 2, Niš, - 18000
|
|
|
|
|