Slide 1
Slide 1 text
Best Practices for Running
Shopware 6 in Production
Benjamin Eberlei
Slide 2
Slide 2 text
Agenda
• Cronjobs and Queues
• Caching
• Http Cache
• Advanced Queues
• Plugins and Performance
• Scale to Multiple Application Servers
• Elasticsearch
2
Slide 3
Slide 3 text
3
Cronjobs and Queues
Slide 4
Slide 4 text
• Shopware 6 uses long-running queues to
process expensive tasks in the background
• Cronjobs are sent as tasks to the queue
• The default is to run these tasks when a user is
logged into the admin (“admin worker”)
Production setup must replace admin worker with
stable, supervised and predictable Linux service
process.
4
Cronjobs and
Queues
Slide 5
Slide 5 text
Cronjobs and Workers
# config/packages/shopware.yaml
shopware:
admin_worker:
enable_admin_worker: false
5
Slide 6
Slide 6 text
Cronjobs and Workers
# /etc/systemd/system/shopware-consumer.service
[Unit]
Description=Shopware Message Queue Consumer
# Default queue backend is based on MySQL
After=mysql.service
[Service]
User=www-data
Restart=always
RestartSec=2
TimeoutStopSec=5s
ExecStart=/usr/bin/php /var/www/shopware/bin/console messenger:consume
$ sudo systemctl daemon-reload
$ sudo systemctl enable shopware-consumer
$ sudo systemctl start shopware-consumer
6
Slide 7
Slide 7 text
Cronjobs and Workers
# /etc/systemd/system/shopware-scheduled-tasks.service
[Unit]
Description=Shopware Scheduled Tasks Worker
# Default queue backend is based on MySQL
After=mysql.service
[Service]
User=www-data
Restart=always
RestartSec=2
TimeoutStopSec=5s
ExecStart=/usr/bin/php /var/www/shopware/bin/console scheduled-task:run
$ sudo systemctl daemon-reload
$ sudo systemctl enable shopware-scheduled-tasks
$ sudo systemctl start shopware-scheduled-tasks
7
Slide 8
Slide 8 text
Cronjobs and Workers
# worker.log
2021-08-15 10:39:58 ProductIndexing data=1791edec1f04464a8a4074423501e781
duration=0.190
2021-08-15 11:30:48 UpdateThumbnails mediaIds=102ac62ba27347a688030a05c1790db7
duration=0.015
2021-08-15 11:32:16 DeleteFile files=shirt_red_600x600_800x800.jpg duration=0.002
2021-08-15 12:08:25 WarmUp route=frontend.detail.page
domain=http://shopware6.tideways.io cache_id=cc3e1493bc874b62b16c11cd291826f9
duration=0.003
2021-08-15 12:08:25 WarmUp route=frontend.navigation.page
domain=http://shopware6.tideways.io cache_id=cc3e1493bc874b62b16c11cd291826f9
duration=0.002
8
Slide 9
Slide 9 text
9
Caching
Slide 10
Slide 10 text
• Different Caches
• Object
• Tags
• HTTP
• System (Symfony)
• Annotations
• Validator
• Serializer
• Property Access / Property Info
• Using a Filesystem based Cache by default
• Expensive and inefficient due to
Shopware’s heavy use of “cache tags”.
10
Caching
Slide 11
Slide 11 text
Caching
# config/packages/framework.yaml
framework:
cache:
default_redis_provider: 'redis://127.0.0.1:6379'
pools:
cache.object:
default_lifetime: 3600
adapter: cache.adapter.redis
tags: cache.tags
cache.tags:
default_lifetime: 3600
adapter: cache.adapter.redis
1
1
Slide 12
Slide 12 text
1
2
HTTP Cache
Slide 13
Slide 13 text
• Caches the complete HTML output of a page
• Can reduce page response times from >=500ms
into the 30-50ms range
• Watch the cache hit ratio: A lot of things can go
wrong
13
HTTP Cache
Slide 14
Slide 14 text
HTTP Cache
use Shopware\Storefront\Framework\Cache\Event\HttpCacheItemWrittenEvent;
use Shopware\Storefront\Framework\Cache\Event\HttpCacheHitEvent;
class HttpCacheHitRatioListener
{
public function onCacheMiss(HttpCacheItemWrittenEvent $event)
{
$redis = $this->getRedis();
$redis->hincrby('http_cache_misses', date('YmdH'), 1);
}
public function onCacheHit(HttpCacheHitEvent $event)
{
$redis = $this->getRedis();
$redis->hincrby('http_cache_hits', date('YmdH'), 1);
}
}
1
4
Slide 15
Slide 15 text
HTTP Cache
15
• Prefer a high cache duration instead of the 2
hours default
• Cache Invalidation for stock/price updates works
pretty well in Shopware 6
• Use bin/console http:cache:warm:up to refresh
cache in the background
• Trigger WarmUpMessage yourself
Slide 16
Slide 16 text
1
6
Advanced Queues
Slide 17
Slide 17 text
• Dispatch your own Messages to the Queue
• Add your own entity indexers
17
Advanced
Queues
Slide 18
Slide 18 text
class ImportFromPimCommand extends Command
{
private MessageBusInterface $messageBus;
protected function execute(InputInterface $input, OutputInterface $output): int
{
$this->messageBus->dispatch(new ImportFromPimMessage());
}
}
class ImportFromPimController
{
private MessageBusInterface $messageBus;
public function importAction(Request $req)
{
$this->messageBus->dispatch(new ImportFromPimMessage($req->getContent()));
}
} 1
8
Advanced Queues
Slide 19
Slide 19 text
use Shopware\Core\Framework\MessageQueue\Handler\AbstractMessageHandler;
class ImportFromPimHandler extends AbstractMessageHandler
{
public function handle($message)
{
if (!($message instanceof ImportFromPimMessage)) {
return;
}
foreach ($message->getRows() as $importRow) {
$this->messageBus->dispatch(new ImportRowFromPimMessage($importRow));
}
}
}
1
9
Advanced Queues
Slide 20
Slide 20 text
# config/services.yaml
services:
"App\\Messenger\\ImportFromPimHandler":
arguments: ["@message_bus"]
tags:
- name: messenger.message_handler
handles: "App\\Messenger\\ImportFromPimMessage"
2
0
Advanced Queues
Slide 21
Slide 21 text
$stockUpdates = [];
foreach ($payload['data'] as $product) {
$stockUpdates[] = ['id' => $product['id'], 'stock' => $product[‘stock’]];
}
$response = $client->post(
'https://example.com/api/_action/sync',
['body' => json_encode([
'write-product' => [
'entity' => 'product',
'action' => 'upsert',
'payload' => $stockUpdates,
]
]),
'headers' => [
'indexing-behavior' => 'use-queue-indexing',
]
);
2
1
Advanced Queues
Slide 22
Slide 22 text
Advanced Queues
2
2
class MyCustomStockIndexer extends EntityIndexer
{
public function getName(): string
{
return "my_custom_stock_indexer";
}
}
# config/services.yaml
services:
"Shopware\\Production\\Indexer\\MyCustomStockIndexer":
tags:
- name: shopware.entity_indexer
priority: 500
Slide 23
Slide 23 text
Advanced Queues
2
3
class MyCustomStockIndexer extends EntityIndexer
{
function update(EntityWrittenContainerEvent $event): ?EntityIndexingMessage
{
$updates = $event->getPrimaryKeys(ProductDefinition::ENTITY_NAME);
if (count($updates) === 0) {
return null;
}
return new MyCustomStockIndexingMessage(array_values($updates));
}
}
Slide 24
Slide 24 text
Advanced Queues
2
4
class MyCustomStockIndexer extends EntityIndexer
{
public function handle(EntityIndexingMessage $message): void
{
$ids = $message->getData();
$stocks = $this->queryStockFromRepository($ids);
$this->redis->hmset('product_stocks', $stocks);
}
}
Slide 25
Slide 25 text
• What if the worker cannot process tasks fast
enough?
• Monitoring and Alerting based on the
backlog
• Database row locks can cause scalability issues
• Shopware “message_queue_stats”
• Use Redis or RabbitMQ as queue instead of the
Database
25
Advanced
Queues
Slide 26
Slide 26 text
Advanced Queues
# config/packages/enqueue.yaml
enqueue:
redis:
transport:
dsn: "redis+phpredis://127.0.0.1:6379"
client: ~
# config/packages/framework.yaml
framework:
messenger:
transports:
default:
dsn: "enqueue://redis"
2
6
Slide 27
Slide 27 text
Advanced Queues
# /etc/systemd/system/[email protected]
[Unit]
Description=Shopware Message Queue Consumer (%i)
# Default queue backend is based on MySQL
After=mysql.service
PartOf=shopware-consumers.target
[Service]
User=www-data
Restart=always
RestartSec=2
ExecStart=/usr/bin/php /var/www/shopware/bin/console messenger:consume
2
7
Slide 28
Slide 28 text
Advanced Queues
# /etc/systemd/system/shopware-consumers.target
[Unit]
Description=Shopware Message Queue Consumers
[email protected] [email protected]
[Install]
WantedBy=multi-user.target
$ sudo systemctl enable shopware-consumers.target
$ sudo systemctl start shopware-consumers.target
2
8
Slide 29
Slide 29 text
2
9
Plugins and Performance
Slide 30
Slide 30 text
Plugins and
Performance
1. Performance Baseline for Shopware 6
2. Core Performance: Update to 6.4
3. Plugin Performance
30
Slide 31
Slide 31 text
Plugins and Performance
Baseline Performance
What range of performance can we expect from Shopware 6?
3
1
Page Type Lowest 95% Percentile Highest 95% Percentile
Category 657 ms 5,983 ms
Product 713 ms 2,533 ms
Slide 32
Slide 32 text
Plugins and Performance
Core Performance: Update to Shopware 6.4
Version 6.4 of Shopware was extensively optimized for better performance.
• Much faster Caching API
• Improves performance of every part of Shopware
3
2
Slide 33
Slide 33 text
33
Slide 34
Slide 34 text
Plugin: EU Cookie Richtlinie Pro
Accidental use of uncached translation API costs 30-60 ms in every request
• Identify by looking at listeners called by Event Dispatcher
• Plugin Authors Acris Ecommerce immediately responded to patch with
new release
3
4
Plugins and Performance
Slide 35
Slide 35 text
35
Slide 36
Slide 36 text
3
6
Scale to Multiple Application Servers
•
Slide 37
Slide 37 text
• Share JWT Private Key, Certificate between App
Servers
• Store Files on AWS S3 or other file storage
service
• Deployment Process
• Update Code to new Version on all Servers
• Point Webserver to new directory
• Restart all workers
• Store Sessions in Redis
• Install Plugins via Composer
37
Scale to
Multiple
Servers
Slide 38
Slide 38 text
# config/packages/framework.yaml
framework:
session:
handler_id: 'redis://127.0.0.1:6379'
3
8
Scale to Multiple Servers
Slide 39
Slide 39 text
Scale to Multiple Servers
3
9
# composer.json
{
"repositories": [
{
"type": "composer",
"url": "https://packages.friendsofshopware.com/"
}
]
}
Slide 40
Slide 40 text
4
0
Elasticsearch
•
Slide 41
Slide 41 text
• For stores with a lot of products Elasticsearch is
recommended
• Elasticsearch Integration uses indexers and
integrates with the Message Queue
• Integrates with everything we talked about
41
Elasticsearch
Slide 42
Slide 42 text
• Blog Posts about Shopware 6 Performance and
Production Best Practices
https://tideways.com/profiler/blog
• Contact me via email
[email protected]
• Contact me in Shopware Slack @beberlei
42
Questions?