Et pourtant le test passait sur ma machine!

Docker est devenu un outil incontournable pour simplifier et optimiser le développement et déploiement d’applications. Dans ce talk, je vous montre comment tirer parti du même principe pour rendre les tests d’intégration plus robustes grâce aux testcontainers.
Node.js Paris Octobre 2024

Transcript

1. Et pourtant le test passait sur ma machine! Bedis El Acheche - Node.js Paris - Octobre 2024 Introduction aux
2. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
3. $ whoami Tech Lead @ bedis-elacheche bedis-elacheche _bedis bedis@furtek.fr Bedis EL ACHECHE Lead Developer Fullstack
6. Tests d’intégration • Vérifient et garantissent que les différents services fonctionnent ensemble comme prévu. • Sont cruciales pour détecter les problèmes qui surviennent non pas au sein des unités isolées, mais dans l'interaction entre elles, en préservant l'intégrité du produit final. • Comparés aux tests unitaires, ils sont plus complexes et plus longs à mettre en place et à exécuter.
7. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
8. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
9. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
10. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27017/ms-users`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => { describe("(GET)", () => { it("returns an empty collection", () => { return request(app.getHttpServer()) .get("/users") .expect(HttpStatus.OK) .expect([]); }); }); }); });
11. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27017/ms-users`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => { describe("(GET)", () => { it("returns an empty collection", () => { return request(app.getHttpServer()) .get("/users") .expect(HttpStatus.OK) .expect([]); }); }); }); });
12. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27017/ms-users`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => { describe("(GET)", () => { it("returns an empty collection", () => { return request(app.getHttpServer()) .get("/users") .expect(HttpStatus.OK) .expect([]); }); }); }); });
13. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => { describe("(GET)", () => { it("returns an empty collection", () => { return request(app.getHttpServer()) .get("/users") .expect(HttpStatus.OK) .expect([]); }); }); }); });
14. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => { describe("(GET)", () => { it("returns an empty collection", () => { return request(app.getHttpServer()) .get("/users") .expect(HttpStatus.OK) .expect([]); }); }); }); });
15. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => { describe("(GET)", () => { it("returns an empty collection", () => { return request(app.getHttpServer()) .get("/users") .expect(HttpStatus.OK) .expect([]); }); }); }); });
17. Testcontainers Une bibliothèque open source permettant de fournir des instances jetables de: • Bases de données • Messages brokers • Navigateurs Web • À peu près tout ce qui peut s'exécuter dans un conteneur Docker. • … Même des Dockerfile ou des Docker Compose!
20. Workflow
21. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
22. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); }); pnpm i —-save-dev testcontainers
23. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
24. describe("UserController", () => { let app: INestApplication; afterAll(async () => { await mongoose.disconnect(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://localhost:27018/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); }); let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer('mongo:6') .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); await container.stop();
25. ${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer('mongo:6') .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb:// MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
32. 6') describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer(‘mongo: .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
33. 6') describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer(‘mongo: .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
34. 8’) describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer(‘mongo: .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
35. describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer(‘mongo:8') .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
36. describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new GenericContainer(‘mongo:8') .withExposedPorts(27017) .withWaitStrategy(Wait.forLogMessage('Waiting for connections')) .start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); }); pnpm i —-save-dev @testcontainers/mongodb
37. describe("UserController", () => { let app: INestApplication; let container: StartedTestContainer; beforeAll(async () => { container = await new MongoDBContainer(‘mongo:8').start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot(`mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`), MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); });
38. describe("UserController", () => { let app: INestApplication; let container: beforeAll(async () => { container = await new MongoDBContainer(‘mongo:8').start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot( MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); }); StartedMongoDBContainer; `mongodb://${container.getHost()}:${container.getMappedPort(27017)}/ms-users-test`),
39. describe("UserController", () => { let app: INestApplication; let container: beforeAll(async () => { container = await new MongoDBContainer(‘mongo:8').start(); }); afterAll(async () => { await mongoose.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [ MongooseModule.forRoot( MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]), ], controllers: [UserController], providers: [UserService], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe("/users", () => {...}); }); StartedMongoDBContainer; `${container.getConnectionString()}/ms-users-test`, { directConnection: true }),
40. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
41. describe('ConversationController', () => { let container: StartedTestContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new GenericContainer('redis:7.2-alpine') .withExposedPorts(6379) .withWaitStrategy(Wait.forLogMessage('Ready to accept connections')) .start(); store = await redisStore({ url: `redis://${container.getHost()}:${container.getMappedPort(6379)}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); });
42. describe('ConversationController', () => { let container: StartedTestContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new GenericContainer('redis:7.2-alpine') .withExposedPorts(6379) .withWaitStrategy(Wait.forLogMessage('Ready to accept connections')) .start(); store = await redisStore({ url: `redis://${container.getHost()}:${container.getMappedPort(6379)}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); });
43. describe('ConversationController', () => { let container: StartedTestContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new GenericContainer('redis:7.2-alpine') .withExposedPorts(6379) .withWaitStrategy(Wait.forLogMessage('Ready to accept connections')) .start(); store = await redisStore({ url: `redis://${container.getHost()}:${container.getMappedPort(6379)}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); }); pnpm i —-save-dev @testcontainers/redis
44. describe('ConversationController', () => { let container: StartedRedisContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new RedisContainer('redis:7.2-alpine').start(); store = await redisStore({ url: `${container.getConnectionUrl()}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); });
45. describe('ConversationController', () => { let container: StartedTestContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new GenericContainer('redis:7.2-alpine') .withExposedPorts(6379) .withWaitStrategy(Wait.forLogMessage('Ready to accept connections')) .start(); store = await redisStore({ url: `redis://${container.getHost()}:${container.getMappedPort(6379)}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); });
46. describe('ConversationController', () => { let container: StartedTestContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new GenericContainer(' .withExposedPorts(6379) .withWaitStrategy(Wait.forLogMessage('Ready to accept connections')) .start(); store = await redisStore({ url: `redis://${container.getHost()}:${container.getMappedPort(6379)}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); }); redis:7.2-alpine')
47. describe('ConversationController', () => { let container: StartedTestContainer; let store: RedisStore; let cacheService: Cache; let app: INestApplication; beforeAll(async () => { container = await new GenericContainer(' .withExposedPorts(6379) .withWaitStrategy(Wait.forLogMessage('Ready to accept connections')) .start(); store = await redisStore({ url: `redis://${container.getHost()}:${container.getMappedPort(6379)}/0`, }); cacheService = createCache(store); }); afterAll(async () => { await store.client.disconnect(); await container.stop(); }); beforeEach(async () => { const moduleFixture: TestingModule = await Test.createTestingModule({ providers: [ ConversationService, { provide: CACHE_MANAGER, useValue: cacheService }, ], controllers: [ConversationController], }).compile(); app = moduleFixture.createNestApplication(); await app.init(); }); describe('/conversations', () => {...}); }); valkey/valkey:8')
48. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
49. Use case Application de messagerie éphémère Gateway Graphql MongoDB ms-users Redis ms-chat
50. services: db: image: mongo:8 networks: - proxy ports: - ${MONGO_PORT}:27017 volumes: - db-data:/data/db redis: image: redis:7.2-alpine networks: - proxy ports: - ${REDIS_PORT}:6379 volumes: - redis-data:/data ms-users: environment: DB_URL: mongodb://db:27017/ms-users build: context: . dockerfile: Dockerfile target: ms-users image: ms-users networks: - proxy depends_on: - db ports: - ${MS_USERS_PORT}:3000 volumes: db-data: driver: local redis-data: driver: local networks: proxy: driver: bridge ms-chat: environment: KV_STORE_URL: redis://redis:6379 build: context: . dockerfile: Dockerfile target: ms-chat image: ms-chat networks: - proxy depends_on: - redis ports: - ${MS_CHAT_PORT}:3000 gateway: environment: MS_USERS_API_URL: http://ms-users:3000 MS_CHAT_API_URL: http://ms-chat:3000 build: context: . dockerfile: Dockerfile target: gateway image: gateway networks: - proxy depends_on: - ms-users - ms-chat ports: - ${GATEWAY_PORT}:3000
51. // setup.global.ts export default async function globalSetup(globalConfig: Config.GlobalConfig, projectConfig: Config.ProjectConfig) { const [REDIS_PORT, MONGO_PORT, MS_USERS_PORT, MS_CHAT_PORT] = await getRandomAvailablePorts({ take: 4 }); const environment = await new DockerComposeEnvironment(`${__dirname}/../../../`, 'docker-compose.yml') .withEnvironment({ REDIS_PORT: `${REDIS_PORT}`, MONGO_PORT: `${MONGO_PORT}`, MS_USERS_PORT: `${MS_USERS_PORT}`, MS_CHAT_PORT: `${MS_CHAT_PORT}`, }) .withWaitStrategy('redis', Wait.forLogMessage(/.*ready to accept connections.*/i)) .withWaitStrategy('db', Wait.forLogMessage(/.*waiting for connections.*/i)) .withWaitStrategy('ms-users', Wait.forLogMessage(/.*application running.*/i)) .withWaitStrategy('ms-chat', Wait.forLogMessage(/.*application running.*/i)) .withProjectName('test') .up(['db', 'redis', 'ms-users', 'ms-chat']); const msChatContainer = environment.getContainer('ms-chat-1'); const msUsersContainer = environment.getContainer('ms-users-1'); process.env.MS_CHAT_API_URL = `http://${msChatContainer.getHost()}:${msChatContainer.getFirstMappedPort()}`; process.env.MS_USERS_API_URL = `http://${msUsersContainer.getHost()}:${msUsersContainer.getFirstMappedPort()}`; globalThis.TEST_CONTAINERS_ENVIRONEMENT = environment; } // teardown.global.ts export default async function globalTearDown(globalConfig: Config.GlobalConfig, projectConfig: Config.ProjectConfig) { await (globalThis.TEST_CONTAINERS_ENVIRONEMENT as StartedDockerComposeEnvironment).down(); }
53. CI/CD
54. Conclusion • Tests en environnement réel avec de vrais services. • Environnements de test isolés, sans interférence avec le système local. • Tests identiques et reproductibles, quel que soit l'environnement. • Configuration simple et flexible de services Docker pour les tests. • Conteneurs Docker avec un démarrage rapide et une gestion automatique. • Modules prêts à l’emploi pour différents services. • Compatibilité avec les pipelines CI/CD. • Grande communauté avec mises à jour régulières.
55. Merci de votre attention bedis-elacheche/nodejs-paris-24-9