This morning I was surprised not only by heavy snowfall but also by an email from my colleague Ralf, who sent me a link to a Heise article and a BSI press release.
The articles concern Magento shops infected with malware.
The topic itself is old news to me (and to Ralf too). Security vulnerabilities in Magento itself are nothing new. For years, we have made a particularly urgent effort to close vulnerabilities immediately when they are disclosed. I do not know about you, but I cannot sleep peacefully if the vulnerabilities are not closed promptly.
By promptly, I mean within 24 hours of the patches being released. And by patching, I do not mean applying and testing the patch in the source code. The patch should of course also go into production. That sounds obvious, but it is not.
Keeping track of many projects
At netz98, I developed a tool for this. We call the tool "Project Control Center". Its purpose is to aggregate data globally from various other systems in one place. One aspect is the patches applied to a system. We read these directly from the "master" branch of the GIT repository.
The whole thing only really gets interesting in combination with a patch database we have set up.
As soon as a patch is released, it always takes priority at netz98. We put all feature development on hold and apply the patch first. Developers and project managers work together to get the patch into the production environment as quickly as possible.
In the Project Control Center, we can then keep an overview of all projects and identify which systems have not yet had a patch applied.
How a patch day works
Since we update all projects at the same time (customers with support contracts first), problems usually become apparent quickly and can be shared directly among the developers in-house. A common problem is that patches often affect Magento's downloader. Since we removed the downloader from shops years ago for security reasons, applying a patch sometimes fails. The patch is adjusted and then made available to the other developers again. We also keep the patches themselves under version control. That keeps everything traceable. In addition, we are known for covering shops with many automated tests. After a patch is applied, the entire test suite for the customer project is started automatically. Potential problems usually become apparent quickly here. Prioritization matters here too. Even if an applied patch causes smaller problems in the shop, the patch is generally still put into production. The motto here is: Safety first
Causes of security vulnerabilities
Now to the patch mentioned by the BSI. At the beginning, I wrote that this is old news. So far, none of our customer projects have been affected. The probability that a customer system will be affected by the malware mentioned in the BSI article is extremely low. As described, we always keep very up to date with patching our systems. In addition, the shop version on the server is often replaced, since we always put new versions into operation through an automated rollout. Even if an attacker manipulated source code on the server, it would not remain there for long, since it would be replaced by a new version at the next deployment.
How do problems like this arise? How can more than 1,000 Magento shops in Germany alone be affected by such problems?
The reason is extremely mundane: shop operators do not budget money for operating and maintaining their e-commerce system. They often cut costs in the wrong places. There is no balance between further development and stabilization.
I have personally seen a lot of online shops like this in the course of analyses. The same picture appears time and again:
- The Magento version is severely outdated
- A large number of third-party modules are installed
- No software version control (e.g. GIT or SVN) is used
- Rollouts are not automated
- The PHP version on the servers is not up to date
- The production environment's operating system is not updated (uptime >300 days and no new Linux kernel patches)
- Improper changes are made to the Magento core
- There is no proper quality assurance
- Module adjustments are made directly in the production system
- There is no staging system for testing things
My conclusion
These are just the things that came to mind straight away. In my opinion, professional e-commerce operations are only possible if all the points here are addressed.
BSI President Arne Schönborn is quoted as follows:
"Unfortunately, it is still apparent that many operators are very negligent when it comes to securing their online shops. A large number of shops run on outdated software versions containing several known security vulnerabilities."
I can only agree with him here. In my view, shop operators should be held much more accountable. Whether penalties are an appropriate measure is a question for policymakers. Professional e-commerce operations are only possible with suitable and experienced staff. Anyone who does not have the necessary resources themselves should purchase professional support. Good Magento agencies can certainly be found with a little research.
That was my ten cents on this topic for now.
I would be interested to hear your opinion…