Asynchronous Communication in E-Commerce

Asynchronous Communication in E-Commerce
General #consumer #message-queue #middleware #serverless #sicherheit #skalierung

I admit that the title is not immediately self-explanatory and sounds a little technical. Yet we all use “asynchronous communication” in everyday life and would not want to do without it.

One example of asynchronous communication is good old email.

As soon as we receive a new email, we can decide for ourselves whether to answer it immediately, leave it for a while, or ignore it altogether. Unlike a direct (non-asynchronous) message—as is common in a normal chat situation, for example—we have a certain freedom of choice as email recipients. This is to the disadvantage of the sender, who does not receive a response to their message immediately, or receives one only after a delay.

But what does this have to do with e-commerce?

Practical Example

E-commerce systems are becoming increasingly complex. Running a “simple” online shop is no longer enough. While providing a shopping cart with a checkout process used to suffice, more and more processes are now necessarily involved in running a shop.

Let us take the example of a customer ordering an item in a shop. What happens in the shop next is not transparent to everyone.

A typical sequence looks something like this:

  • Convert the shopping cart into an order
  • Deactivate the existing shopping cart
  • Send emails (e.g. an order confirmation)
  • Update caches when prices and availability change
  • Update sales statistics
  • Process inventory changes
  • Export order data to an ERP system

So what advantage does asynchronous communication offer? As with email, we can decide in shop systems not to start processing these operations immediately.

The shop system can display a “success page” to the customer immediately after saving the order. The customer therefore does not have to wait until the system has sent the order confirmation by email. Some processes also require longer computation times. An item could, for example, become unavailable in the meantime. Asynchronous communication therefore ensures that the customer has priority. Conversely, an order results in a change in stock, which in turn affects the prices displayed in a shop. However, prices should be updated immediately, rather than only after the order has been fully processed.

Asynchronous communication now allows the resulting tasks to be placed in queues. The system can distribute the tasks for processing at a later time. This gives us a fast shop frontend and avoids boring customers with loading animations.

Technical Solution

So-called “message queues” are often used to implement this kind of asynchronous communication. In most cases, a message consists of a simple instruction or an object to be processed (such as the order to be exported, or an item number and an ordered quantity). The instruction can also be implicit in placement in the relevant queue, if that queue contains only messages for a defined task.

For this purpose, the shop system sends messages to a message broker. In this context, the sending system is also called the “publisher.”

A message broker acts as an intermediary (broker) for messages. Incoming messages are delegated to the corresponding queues according to their topic and a set of rules. In addition, the broker will usually confirm receipt of the message to the publisher. This takes the message out of the shop's context for the time being, allowing the shop to return to its primary task: selling products.

Modern message brokers support a wide variety of protocols. This allows them to connect to many systems.

But what happens to the message now? Ultimately, of course, the order has to be exported or the stock updated.

This is where the so-called “consumer” comes into play. It “consumes” the messages from the queue. The consumer asks the message broker for a certain number of messages from a particular queue. The broker delivers the messages, which the consumer then interprets and processes: the order is exported or the stock is updated.

Modern queue systems provide transactional safety at this point: after the message is processed, it is marked accordingly in the message queue. This guarantees, for example, that an order is exported only once. If the connection breaks, the message is not lost but can be processed again. This also provides another advantage: fault tolerance.

The term “quality of service” is often used here. For online shops in particular, ensuring continued operation is essential to revenue.

Scaling and Security

Scaling is becoming increasingly important in the e-commerce environment. As revenue grows, the enterprise shop platform has to grow with it. If you use a system that scales only through hardware (servers), growth soon reaches its limit.

Asynchronous communication can help us here too.

The queue systems described are inherently scalable. It is possible to run more than one system. Messages can then be distributed across multiple systems.

Fault tolerance is also an important issue here in ensuring service quality. A good overall message queue system can be protected through replication. Messages are then automatically distributed across multiple systems. If one message queue server fails, another takes over its job. This is referred to as a “failover”1.

Of course, not every message is equally important, so messages can be prioritized accordingly.

This ensures that absolutely essential work is always completed first and that critical processes do not stall even under heavy load.

Another way to scale is to increase the number of consumers processing the messages.

Transactional safety allows messages to be distributed to multiple processes that take on work in parallel. This makes it possible to process data efficiently and at high speed in the background.

Middleware

Modern message brokers support a wide variety of protocols. This makes it easy to connect them to many different systems. As middleware, they allow messages to be distributed to more than one recipient. A recipient (consumer) can subscribe to the broker. As soon as the broker receives a message, the message is checked against the rules. If the message is suitable for delivery, it is then assigned to all matching recipients.

This allows a wide variety of third-party systems to communicate with one another through the message broker. The e-commerce system, for example, needs to publish the order data only once. The order can then be processed in an ERP at the same time. It is also possible to feed the data into a CRM to evaluate it as part of customer analysis. Such evaluations often require substantial computing resources.

Outsourcing these calculations to external services is already possible too. Major cloud providers such as Amazon2, Google3, or Microsoft4 directly offer services that can handle all the tasks. This gives you new alternatives for organizing your infrastructure. This logic is often grouped under the buzzword “serverless architecture.” The term is misleading, however, because the servers are of course still there. But you do not have to take care of maintaining and scaling the systems yourself. You simply use the service.

The advantage is obvious. Middleware of this kind centrally links complex e-commerce systems. This gives you a better overview of the overall system again. Decoupling the functions also makes it easier to move them into separate applications. This makes it possible to build a microservice landscape.

This is an advantage that should not be underestimated. Large, long-established systems in particular tend to stumble when they have become too hard to understand. A lot of time is lost troubleshooting.

Standardized middleware solutions can drastically reduce the effort here and enable the business to scale.

Conclusion

Asynchronous communication offers new opportunities for scaling. It also gives you alternatives when building an e-commerce landscape. Complex systems can be made easier to understand. End users also benefit from faster page access and faster services. All these points can have a positive impact on an online shop's revenue.

The article first appeared in netz98 Zukunftsthemen 2017.