SysOperation in D365 F&O: Why My Batch Job Said Ended and Did Nothing

A message from finance: "The nightly cost recalculation did not run last week." I opened batch job history. Every night, status Ended. Green, clean, no log entries. The job had run seven times and done nothing seven times, and the system was proud of it.

It took ten minutes to find the cause and a lot longer to explain it. One record with a missing item reference threw an error on the first night. The service caught it, said nothing, and rolled back the whole run. The same bad record was still there the next night, and the night after.

In the previous article I said that a value which must survive into batch belongs in the data contract. Today I want to go one level deeper into the SysOperation framework itself: how the controller, contract and service split the work, why a batch job can report success while doing nothing, and how to test the logic without waiting for the batch server to pick it up.

THREE CLASSES, THREE JOBS

SysOperation replaced RunBaseBatch for a simple reason: RunBaseBatch mixed everything into one class. The dialog, the parameters, the pack and unpack of those parameters, and the actual business logic all lived together. SysOperation splits that into three pieces, and most problems I debug come from code that sits in the wrong piece.

• The data contract holds parameters and nothing else. Each parm method decorated with DataMemberAttribute becomes a field on the dialog and a node in the serialized XML.

• The controller decides how the operation runs: which service and method, which execution mode, whether a dialog appears, whether it can go to batch. It runs on the client side of the call.

• The service is a class with one public method that takes the contract. This is the only place where business logic should live, and it is the code that actually runs on the batch server.

The diagram shows the part people miss. Between the controller and the service there is a serialization step. The contract is turned into XML, stored with the batch task, and rebuilt later in a different session, possibly on a different server. Anything that is not a data member does not make that trip.

THE CONTRACT: PARAMETERS, NOT STATE

A clean contract looks boring, and that is the point:

[DataContractAttribute]
class InvRecalcContract
{
    TransDate fromDate;
    InventSiteId siteId;

    [DataMemberAttribute, SysOperationLabelAttribute(literalStr("From date"))]
    public TransDate parmFromDate(TransDate _fromDate = fromDate)
    {
        fromDate = _fromDate;
        return fromDate;
    }
    // parmSiteId follows the same pattern
}

The wrong way I see most often is a contract that carries a table buffer, a class instance, or a member variable set in code but not exposed as a data member. In synchronous mode on a developer box it works, because the same object is passed straight through. In batch the service receives a rebuilt contract, the extra variable is empty, and the job quietly processes nothing or everything.

The right way: only primitive types, EDTs, enums, containers and lists through AifCollectionTypeAttribute. If you need a record, pass its RecId and find it again inside the service. If you need validation, implement SysOperationValidatable on the contract so the user gets the error in the dialog, not the batch server at 2 a.m.

One more trap: scheduled recurring jobs keep their serialized contract. Rename a parm method in a later release and every existing recurring job comes back with that parameter empty. I treat data member names like a public API: add new ones freely, never rename the old ones.

THE CONTROLLER: CHOOSE THE EXECUTION MODE ON PURPOSE

The controller decides where the work happens. Synchronous blocks the user and runs in their session. ReliableAsynchronous sends the work to the batch framework and gives the user their screen back, with batch history you can read later. ScheduledBatch is the classic batch job with recurrence. Plain Asynchronous is the one I avoid for anything important, because when it fails there is very little to look at.

My rule: if it takes longer than a few seconds or touches more than a few hundred records, it is ReliableAsynchronous or batch. The controller is also where I override canGoBatch, set a sensible caption, and decide whether to show the dialog at all. It is not where I put queries, loops or postings. If logic sits in the controller, it cannot be tested on its own and it does not run in batch the way you expect.

THE SERVICE: WHY THE JOB SAYS ENDED

Here is the service from that Monday morning, simplified:

public void process(InvRecalcContract _contract)
{
    try
    {
        ttsbegin;
        while select forupdate line where line.TransDate >= _contract.parmFromDate()
        {
            this.recalc(line);   // one bad record throws here
        }
        ttscommit;
    }
    catch
    {
        // "we will look at it later"
    }
}

Three things go wrong at once. The whole run is one transaction, so a single bad record rolls back hundreds of good ones. The catch block is outside the transaction, so it does run, and it swallows the exception. And because no exception leaves the method, the batch framework has nothing to report. The task status is decided by one question: did an exception escape the service method? Here it did not, so the answer is Ended.

There is a second, subtler trap: inside a ttsbegin block, most exceptions do not stop at the nearest catch. They jump to the catch that is outside the outermost transaction. Developers put a careful try/catch inside the loop, inside the transaction, and wonder why it never fires. Only update conflicts and duplicate key exceptions have special handling there, and even those need the retry logic written correctly.

ISOLATE EACH RECORD, THEN TELL THE TRUTH

The right way keeps the transaction small and the reporting honest:

while select RecId from line where line.TransDate >= _contract.parmFromDate()
{
    try
    {
        ttsbegin;
        this.recalc(InvRecalcLine::findRecId(line.RecId, true));
        ttscommit;
        done++;
    }
    catch (Exception::Error)
    {
        failed++;
        error(strFmt("Line %1 skipped, see message above", line.RecId));
    }
}
if (failed) { throw error(strFmt("%1 lines done, %2 failed", done, failed)); }

Each line commits or rolls back on its own. The catch now sits outside each small transaction, so it really runs. The error is written to the infolog with the record key, and in batch that infolog becomes the task log the support team can open from batch job history. At the end, if anything failed, the service throws. The good records are already committed, and the batch status turns to Error, which means the alert that someone configured on batch job failures finally has something to say.

Whether the good records should commit when some fail is a business decision, not a technical one. For a cost recalculation, partial progress is fine. For a posting that must balance, it is not, and then one transaction for the whole set is correct and the job must fail loudly. What is never correct is failing quietly.

BIG VOLUMES: SPLIT INTO TASKS

When one service has to handle a hundred thousand lines, I stop looping in a single task. The service reads the scope, cuts it into bundles (by site, by date range, by a RecId range) and adds a runtime task per bundle with BatchHeader::addRuntimeTask. Each bundle gets its own contract and its own log. The batch server runs them in parallel across threads, and a failure in one bundle shows up as one red task instead of one mysterious overnight run.

Two cautions. Bundles must not compete for the same rows, or you have built a deadlock generator. And the parent task must not wait for its children by sleeping in a loop; let the batch framework handle the dependency.

TESTING WITHOUT THE BATCH SERVER

Because the logic lives in the service and the service takes a plain contract, a unit test can call it directly. No dialog, no batch server, no waiting for the next pickup cycle:

[SysTestMethod]
public void recalcSkipsBadLineAndReportsIt()
{
    InvRecalcContract contract = new InvRecalcContract();
    contract.parmFromDate(DateTimeUtil::getToday(DateTimeUtil::getUserPreferredTimeZone()));
    // arrange: insert two good lines and one line with a missing item
    try
    {
        new InvRecalcService().process(contract);
        this.fail("Expected the service to report the bad line");
    }
    catch (Exception::Error) { }
    this.assertEquals(2, this.countRecalculatedLines());
}

This test would have caught the Monday problem before it reached production: the old service passed silently and recalculated zero lines. I also add one test that packs and unpacks the contract through the serializer, which catches the "member not exposed" mistake in seconds.

WHAT GOES WRONG

• Status Ended, nothing happened. A catch without a throw, or an empty query because a contract value did not survive serialization. Open the task log first, then check the query range values the service actually used by logging them at the start of the run.

• Job stuck in Executing. Usually a long lock wait or a deadlock with another job on the same tables. Look at SQL insights in LCS for blocking chains, and check whether two recurring jobs were scheduled into the same window.

• Update conflicts on rerun. Records read without forupdate in one place and updated elsewhere, or optimistic concurrency hitting a long loop. Reselect inside the small transaction, as in the right-way snippet.

• Job never starts. The batch group has no server assigned, or the job is waiting on a dependency that ended in Error. Batch job history and the batch group setup answer this faster than any trace.

• Slow but correct. Record-by-record where a set-based update_recordset would do. Capture a trace with the trace parser around one bundle and look at the call count, not just the duration.

The practical takeaway for tomorrow: open your three most important batch services and search for catch. For every block, ask what the batch status will show if that path runs. If the answer is Ended, you have a job that can lie to you.

Next time I will look at business events from the developer side: building a custom business event in X++, what the payload contract should and should not contain, and how to make sure an event is only sent when the transaction that triggered it actually commits.