Tweet
Tweet

Slide 1

Slide 1 text

DevOps for Movie and Cartoon Production OLM Digital, Inc Marc Salvati Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). SA Courses '24, December 03-06, 2024, Tokyo, Japan ACM 979-8-4007-1135-0/24/12. 10.1145/3680532.3689583

Slide 2

Slide 2 text

Disclaimer ● Lots of Dev, little of Ops ● Still learning ● Cannot cover all topics and technologies ● Technologies we talk about ○ Windows/Linux ○ Git/Github/Github Actions ○ CMake ○ C/C++, Python ○ VcPkg ○ Jenkins ○ Maya, After Effects, Arnold, Nuke, Photoshop

Slide 3

Slide 3 text

What is a movie and cartoon production company? ● Produce movies and cartoons ○ Own IP, contractor ○ Use Digital Content Creation (DCC) tools ■ Extends their features using Software Development Kits (SDK) ○ Multiple movie/cartoon projects in parallel ● 90-100% of staff is non IT ○ Some companies have no IT department ○ Machine/servers maintenance outsourced ○ Software engineer in IT companies: 40-60% ○ Production company : 5-10% ● R&D ○ SaaS companies going IPO 23% ○ IT computer services 11 % ○ Movie and cartoon production company up to 5% R&D in Creative Industries Survey 2020 (UK)

Slide 4

Slide 4 text

IT departments in production companies ● System maintenance ○ Servers ○ Network ○ Software licences and installation ● IT support ● Pipeline ● Research and Development More Ops than Dev More Dev than Ops

Slide 5

Slide 5 text

Overview ● Introduction to DevOps in production ● DevOps in production ○ Dev ○ Ops ● Conclusion Code Plan Build Test Release Deploy Operate Monitor DEV OPS

Slide 6

Slide 6 text

Introduction to DevOps in production

Slide 7

Slide 7 text

What is DevOps Wikipedia: DevOps is a methodology in the software development and IT industry. Used as a set of practices and tools, DevOps integrates and automates the work of software development (Dev) and IT operations (Ops) as a means for improving and shortening the systems development life cycle ● Production pipeline ○ Let artists focus on art ● DevOps is the pipeline of IT ○ Let programmers focus on programming ● Pipeline is hard! ○ Calude, Coull, Lewis, Can we solve the pipeline problem? ACM Digipro 2014

Slide 8

Slide 8 text

Code Plan Build Test Release Deploy Operate Monitor DEV OPS

Slide 9

Slide 9 text

● Dev ○ Plan: ■ Study the market: size and target, needs ■ Estimate cost and possible benefits ■ Do it if it will make money ■ Break it down ● Define the project goal, roadmap, milestones ● Organize the work, tickets, Agile ○ Code: program with IDE, source versioning control ○ Build: compile, packaging, multiple OS ○ Test: unit, integration, automated ● Ops ○ Release: prepare to be used by final users, quality assurance ○ Deploy: get to the final user, installer, web services update ○ Operate: customer support, maintenance of servers ○ Monitor: logging and metrics, crash reports General Definition

Slide 10

Slide 10 text

● Dev ○ Plan: ■ Study the market: size and target, needs ■ Estimate cost and possible benefits ■ Do it if it will make money ■ Break it down ● Define the project goal, roadmap, milestones ● Organize the work, tickets, Agile ○ Code: program with IDE, source versioning control, DCC tool plugins ○ Build: compile, packaging, multiple OS, multiple DCC tool versions ○ Test: unit, integration, automated, test in DCC tools ● Ops ○ Release: prepare to be used by final users, quality assurance ○ Deploy: get to the final user, installer, web services update ○ Operate: customer support, maintenance of servers ○ Monitor: logging and metrics, crash reports CG Software company

Slide 11

Slide 11 text

● Dev ○ Plan: ■ Gather requests and frustrations, survey/monitor artists needs ■ Estimate potential time and quality improvement ● Current and future needs ■ Need to fit the schedule ■ Break it down ● Define the project goal, roadmap, milestones ● Organize the work, tickets, Agile ○ Code: program with IDE, source versioning control, DCC tool plugins ○ Build: compile, packaging, multiple OS, multiple DCC tool versions ○ Test: unit, integration, automated, test in DCC tools ● Ops ○ Release: test by developer and selected artists, quality assurance ○ Deploy: ■ Installer ■ Setup DCC tool so plugins may be used (no installer) ■ Dev, Test, production environment ○ Operate: maintenance of servers, artist feedback, render farm, DCC licences ○ Monitor: logging and metrics, crash reports Movie production company

Slide 12

Slide 12 text

A production company / A CG software company ● OLM ○ 328 employees ○ IT 21 ■ System: 4, Research: 2, Pipeline: 4 ■ Development: 9 permanent + 4 interns ● Ephere ● 8 employees ○ 4-5 software engineers? ● Currently support only Ornatrix Development projects (excluding pipeline) DCC Active Support Inactive DCC Versions OS Maya 3 30 57 4 2 Arnold 3 18 8 4 2 AE 3 25 26 3 1 Nuke 1 6 13 2 2 Photoshop 1 3 - 2 1 Standalone 2 3 - 2 DCC Active Support Inactive DCC Versions OS Maya 1 - 9 3 Max 4 9 1 C4D - 7 2 Unreal Engine 1 2 3 Tools: Active 13, Support 75, Tool versions: 3~4 Targets: Active 67 x 3~4 Support 495 x 3~4 Tools: 1, Tool Version:3~4 Targets: Active + Support 56 x 3~4

Slide 13

Slide 13 text

Conclusion ● The business of production companies is NOT software ○ If the picture right then it's right ○ Everything can be done without inhouse development ● Clients = Artists ○ Easy access ○ Easy to lose trust ○ Low IT literacy ● Exotic platforms ○ DCC tools + SDKs ○ Plugins and scripts ○ Specific builds, tests, deploy ● Many tools and target platforms ○ Combination of OS and DCC tools Scale Specific Communication

Slide 14

Slide 14 text

About other approaches ● NoOps/AiOps ○ Release/Deployment: Full automation ○ Monitor: Use AI to collect and analyse data, send feedback ○ Operation: separate from the Dev team ● DevSecOps ○ Don't forget about security in the DevOps approach ● Platform engineering ○ Integrate Agile concept into the DevOps implementation ● SRE (Site Reliability Engineering): ○ Implementation of DevOps ○ More automation? but DevOps is about automation

Slide 15

Slide 15 text

Dev in production

Slide 16

Slide 16 text

Overview ● Type of development in production company ○ Pipeline ○ Research and Development ● Plan ● Code ● Build ● Test

Slide 17

Slide 17 text

Pipeline ● Artists daily support ● Automation ○ Within on step ● Asset management ○ One step to the next ● Save time for quality ● Mostly python scripting 2D 2D Layout Key animation In-betweens 2D Clean-up Compositing 3D Modeling Rigging Animation Lighting Rendering FX Matte Painting

Slide 18

Slide 18 text

Research and Development ● Create new possibilities ○ Visual ○ Functional ● Wide range of development ○ Standalone applications ○ Automation scripts ○ DCC tools plugins ○ DCC tools scripts ○ Server based applications ○ DevOps ● C/C++, python

Slide 19

Slide 19 text

Dev: Plan

Slide 20

Slide 20 text

Plan ● Organize the tasks ○ Define ○ Priority ○ Track ● Software support ○ Text, spreadsheets (Word, Excel, Google apps…) ■ Linear representations ○ Web application (Trello, Redmine, Jira, Github projects…) ■ Easy to search ■ Easy to share ■ Connection with other tools (Github..) ■ Often uses 2D representations ■ May be associated to code ■ More or less customization ● Work Agile?

Slide 21

Slide 21 text

Plan in production ● Artists cannot write IT specifications ● When there is time/money (2-5 years production) ○ Identify development needs during pre production ○ Develop and deliver tools before production starts ● When there is no time/money (6 month- 1 year production) ○ Maybe no need right now, but when they need it it's too late ○ Identify development needs ■ During production and across productions ● Keep communication channels open ■ Synthesis of needs ● Select the one that can help most productions ■ Keep track in the backlog/global backlog ○ Develop and deliver to test at a production that may use it

Slide 22

Slide 22 text

Task status ● Backlog ○ Recent request or idea ○ May move to the global backlogs or pending ● Pending: waiting to be assigned to someone ● Todo: assigned to someone ● Doing: the assigned person is actively working on it ○ What to do when switched task and not working recently on it? ■ Granularity? Other status? ■ Is it that important to not go back? ● Done: implemented but not released yet ● Released ○ Artists can use it ○ Waiting for feedback (may be after project ends) ● Archived: only once we got the feedback and created tickets if needed

Slide 23

Slide 23 text

The global backlogs ● Existing project ○ Current issues and idea ○ To tackle when we have time or need for that project ● Non existing project ○ Conference paper reference ○ Production problem to tackle ○ Organized by topic ● Participate/survey conferences and journals ○ Siggraph / Siggraph Asia ○ Digipro ○ Eurographics ○ High Performance Graphics ○ Computer graphics forum

Slide 24

Slide 24 text

Feedback ● In-house client ○ Feedback loop track ● Client is usually not prone to use tracking systems directly ● Link between user request docs and implementation specs complicated ○ Get docs, chat comments… anything ○ Receive excel sheets, pdfs, slides from various client in various forms ○ Manually linking

Slide 25

Slide 25 text

Dev: Plan About Agile

Slide 26

Slide 26 text

About Agile: The Agile Manifesto We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. Trust Communication Adapt

Slide 27

Slide 27 text

About Agile: Principles behind the Agile Manifesto Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. The best architectures, requirements, and designs emerge from self-organizing teams. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Business people and developers must work together daily throughout the project. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Trust Communication Adapt

Slide 28

Slide 28 text

About Agile: Principles behind the Agile Manifesto Working software is the primary measure of progress. Simplicity--the art of maximizing the amount of work not done--is essential. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Continuous attention to technical excellence and good design enhances agility. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Refactor CI/CD

Slide 29

Slide 29 text

Summary of Agile ● What is there ○ Trust, communication, Adapt ○ Refactor ■ Code ■ Organization ○ Automation to focus on work ⇦ DevOps needed ■ Continuous delivery ● What is not there ○ Scrum, Kanban, Scrum master, product owner… ○ Metrics, granularity, ticket speed, daily meeting… ● Going Agile or not, DevOps is needed! ● Agile for devops in production ○ Because lots of issues arise during production ○ Done when done ■ No deadlines ○ Regular meeting and roadmaps ○ Constant update of time estimation Depend on your Agile implementation Not in Agile itself

Slide 30

Slide 30 text

Scrum and Kanban in IT ● Like processes ○ Role and responsibilities ○ Process and tools ■ Sprint ■ Meeting ■ Tables ○ Stories and point, evaluation ■ Granularity of tasks ■ Ticket flow optimization ■ Dangerous link to evaluation (quantity vs quality) ● Kanban comes from car industry ○ Repetitive tasks, stock management ● The differences between Agile principles and its implementation… ○ There is no scrum master in agile manifesto ■ Scrum is like a war room situation ○ Agile 11th principle: The best architectures, requirements, and designs emerge from self-organizing teams.

Slide 31

Slide 31 text

Dev: Code

Slide 32

Slide 32 text

Code ● Programming rules ○ Naming conventions ○ Coding conventions for each programming language ● Enforces programming rules ○ Code reviews ○ Automatic check ● Efficient coding ○ Use IDE/debugger/profilers ● Source versioning control ○ About Git ● Semantic versioning

Slide 33

Slide 33 text

Programming rules ● Important to have a rule, which rule is less important ● Naming conventions ○ PascalCase, camelCase, snake_case, kebab-case… ■ Per programming word type (class, variable…) ○ Be consistent ● Coding conventions ○ Which feature you use or not, how to use them ■ Ex: exceptions or not? std::expected? Smart pointer? Branchless programming? C++ modules? ○ Code layout for readability ■ Logical structure, comments ● Specific to each programming language ● May vary per project ● Examples ○ Google C++ Style Guide (no exceptions) ○ C++ Core Guidelines (use exceptions) ○ Python PEP 8 (standard)

Slide 34

Slide 34 text

Naming conventions in production ● Each DCC tool SDK has its own conventions ○ Maya C++ class method = camelCase ○ Nuke C++ class method = mix of camelCase and snake_case ○ After Effects C functions = PascalCase ● Two choices ○ Adapt to the convention of each DCC tool ■ What about your libraries? ■ Many rules to follow ■ Not always well defined (Nuke) ○ Use a unique convention ■ Simple ■ Easy to spot your code from the SDKs code ■ Same convention for libraries

Slide 35

Slide 35 text

Coding conventions in production ● May vary with the DCC tool ○ DCC tool have a recommended compiler ■ May limit the features you can use, ex: need vc2015 for c++11, ■ Reference for C++ standard ○ AE mainly C, use error code, own memory management system ■ Would need a wrapping to use exceptions ■ Need custom allocator to use C++ STL ○ Maya use C++, use error code, no exceptions ○ Arnold use C ● Need a base coding conventions ○ Use standard one ■ C++: C++ Core Guidelines ■ Python: PEP8 ○ Make your own for each DCC tool ● Create new conventions for specific languages ○ DCC tool have specific language embedded ■ Maya MEL and expressions, Nuke Blink Scripts, Houdini VEX ○ Exotic languages: OSL, GLSL, TCL

Slide 36

Slide 36 text

Enforcing rules ● C/C++/Java/Javascript: Clang Format, Clang Tidy ● Python: pep8, autopep8 ● Automate process ○ At push to Version Control Software (VCS) ○ At build time ● Code reviews ● Hard to automate anything on exotic languages ○ Maya Mel, Houdini VEX…

Slide 37

Slide 37 text

● Code review = taking time to look at someone else code and talk about it ○ Encourage discussion about code and algorithms ○ Improve quality of code ○ Help to find bug, optimization, refactoring possibilities ○ Helps to enforce coding rules ● Pull requests ○ Provided by cloud git hosts ○ Pros ■ Systematic code review, someone checks! ■ Organize push in bunch ○ Cons ■ Wait before the merge ● Need someone able to check the pull request ○ Easy if pool of many specialized people available ○ Easy if many people work on the same part of the code ■ But then they already saw the code? ○ Hard if you need to get into the code, study it before validation… ● Change in the meantime, complicates the merges ■ Systematic code review may lower its quality? Code reviews

Slide 38

Slide 38 text

Code reviews in production ● Use pull requests? ○ Often one developer on one project ○ Not enough staff ○ Drawbacks may outweigh the benefits ● Code reviews ○ A must ● Make people work overlap ○ Need of two pair of eyes ○ Alternate the programmer on each tool ● Trunk based development ○ Frequent push ○ Continuous integration ■ Automated compilation ■ Automated tests Works well together Works well together

Slide 39

Slide 39 text

Dev: Code Coding tools

Slide 40

Slide 40 text

Efficient coding tools ● IDE ○ Visual studio/ Visual studio code ○ Eclipse, Qt Creator… ○ XCode ● Debugger ○ Integrated to the IDE ○ GDB ● Profiler ○ VTune (Windows/Linux) ○ Valgrind (Linux) ○ Instruments (Mac)

Slide 41

Slide 41 text

Efficient coding tools in production ● Plugins are programs executed in host applications ○ Applications we don't have the source code for ● Debugger ○ Attach the debugger to the DCC tool/host application process ● Profiler ○ DCC tools are not instrumented, hard to profile ○ Some DCC tool have their own profiler ■ Maya profiler, DG profiler ■ AE render time evaluation ■ Nuke profile node ■ Houdini performance monitor

Slide 42

Slide 42 text

Getting answers when stuck ● General c++ ○ C++ reference ○ Google search>>stack overflow ○ ChatGPT, AI? Not used yet… ● Production ○ Dedicated forum ■ Some need an account to search (free) ○ After Effects forum ○ Arnold ○ Houdini ○ Maya ○ Nuke

Slide 43

Slide 43 text

Dev: Code Source versioning

Slide 44

Slide 44 text

Version Control Software (VCS) ● Also called Source Control Management (SCM) ● Features ○ Keep track of every change to code ○ Rollback changes ○ Merge changes ■ Multiple programmer programming in parallel ○ Centralized or decentralized ● Examples ○ Git (de facto standard Market share 89%) ○ Subversion ○ Mercurial ● Can be hosted in the cloud ○ Offers Continuous Integration solution ○ Examples ■ Github ■ Gitlab

Slide 45

Slide 45 text

About Git ● Need to have rules ● Git configuration ○ Merge or rebase ○ LFS handling ■ What file to upload ○ Git ignore ● Git flow VS Trunk based development ○ Git flow ■ Good for open source software ○ Trunk based development ■ Simpler ○ In the reality use a mix of branches ■ Main/Develop ■ Feature ■ Hotfixes ■ Release

Slide 46

Slide 46 text

Trunk based development ● Commit often ○ Atomic commits ○ Tag+branch for release ● Good for senior developers ● Good with CI Main v1.0.x 1.0.0 1.0.1 v1.1.x 1.1.0 1.1.1 Head Merge Bug Fix Bug Fix

Slide 47

Slide 47 text

Git flow ● Develop in develop and tag in main ● Feature branches ○ When need to test some big changes ○ Isolate feature influence on next release ● Release branches ○ When close to release state ● Hotfix branches ○ When problems occurs in the release Main Head Develop

Slide 48

Slide 48 text

● Isolate some development from the main branch ● Merge when ready ● Use pull requests to merge ● Good for large scale change ● Good for experimenting new designs Feature branch Main/Develop Feature 1 Head Merge Feature 2

Slide 49

Slide 49 text

Release branch ● Buffer between main and develop ● Develop can still go on with new features Main Head Trunk based Merge Main Head Develop Release Git flow

Slide 50

Slide 50 text

Hot fixes branch ● Problem in a release, fix it and merge in develop and release ● Develop can still go on with new features Main Head Develop Hotfix Main Head Git flow Trunk based Merge

Slide 51

Slide 51 text

Git branching in production ● Hotfixes branch ○ Merging a bugfix from an older version is usually hard ○ Need specific port usually ○ Hotfixes branches may not make it ● Release branch ○ One branch for one released is practical ○ Hotfixes directly in those branches ● Feature branches ○ May be used when multiple developers on the same project ● Separation of main and develop ○ A matter of preference ○ Personally i prefer the trunk based approach ■ Different version of the tools used at the same time ● Need to patch the released version often ○ Change version in the middle of a production may change output ● One branch per release, hotfix directly in the release branch ■ Branch naming convention is simple

Slide 52

Slide 52 text

Dev: Code Tool versioning

Slide 53

Slide 53 text

Tool version names ● Many patterns ○ Numbers separated by dots: 1.2.3 ○ Add - for specific version ■ -a = alpha ■ -b = beta ■ -rc = release candidate ○ Number order: 8<9<80<90 or 8<81<9<92 ● Internal version number VS release number ○ Internal version number may not be marketable ● Exotic pattern ○ TeX new version adds a new Pi number (current version 3.141592653) ○ Odd number = development, even = release

Slide 54

Slide 54 text

Tool version names ● Some standardization ○ Dot separated numbers recommended ● Semantic versioning ○ Makes a lot of sense for libraries, less for softwares ○ Most software companies have their own schemes and rules 2 1 3 . . API non compatible change Backward compatible feature Backward compatible bug fixes

Slide 55

Slide 55 text

Tool versioning in production ● DCC tool and tool version may be set for 3-5 years ○ High budget production ○ TV series lasting for more than 2 seasons ● What the artists want to know about the new version of a tool? ○ Can i open my files? ○ Is the output picture the same?

Slide 56

Slide 56 text

Semantic versioning for production 2 1 3 2 . . . Data (API) non Compatible change Output Change Data compatible Output stable feature/bug fixes Production specific Hotfixes (usually data compatible and output stable ) Error when opening file Rendering changes Anything else Anything may happened! Production surprise

Slide 57

Slide 57 text

Version 1.0.0.1 The patch number Project A Tool A Version 1 Version 2 Version 2.0.1 Start with Tool A version 1 Find bug with crash Correction in version 2.0.1 Encounter crash Impossible to upgrade to version 2 Create patch version 1.0.0.1 Time Projects end Throw away the patch

Slide 58

Slide 58 text

Dev: Build

Slide 59

Slide 59 text

Build definition Code Internal Library Code External Library Code Internal Library Binary External Library Binary Compiled Code Executable Code Compile Link Plan Test Install …

Slide 60

Slide 60 text

Build overview ● We focus on C/C++ ○ Mainly used in production ● Build ○ Compile ○ Linking ○ About API and ABI ● Build system ○ Build + install + test +... ○ May be generated ● Dependencies ○ Diamond ○ ABI problem ● Automation and continuous integration

Slide 61

Slide 61 text

Build ● Compile and link ○ Windows: Visual c++, MingGW ○ Linux: gcc ○ Mac: clang ● Many flags ○ Optimization level ○ Warning level ○ Features ■ C++ language standard (11, 14, 17, 20) ■ SSE, AVX ■ OpenMP ● Libraries ○ Shared or static ○ C++ runtime/standard library ■ Usually mixing static and dynamic runtime is not a good idea

Slide 62

Slide 62 text

Build in production ● Compile and link ○ All plugins are shared libraries ○ Compiler specified by the DCC tool ○ C++ runtime libraries ■ Dynamic runtime ■ Reuse the c++ dynamic runtime the DCC tool already installed ● Flags ○ Some are specific to the DCC tool SDK ○ Features ■ C++ language standard (11, 14, 17, 20) ● Minimum requirement depending on the SDK ■ OpenMP ● Maya ○ Need to use syncNumOpenMPThreads if using non intel openmp ○ Maya recommends to use Tbb instead ● After effects has its own parallelization functions but can use OpenMP anyways

Slide 63

Slide 63 text

Dev: Build ABI and API

Slide 64

Slide 64 text

About ABI and API breaks ● API: Application Programming Interface ○ Write a program using an API ○ API stable >> source code portable ■ can be compiled against new version without change ○ Non stable API >> has to change the code >> API break ● ABI: Application Binary Interface ○ Data layout, call conventions… ○ Write a program using an API ○ Compile to machine code ○ Library ABI = compiler ABI + library API and data layout ■ Non stable API >> Non Stable ABI >> machine code is not portable ■ Stable API + Stable ABI >> machine code is portable ■ Stable API + Non stable ABI >> machine code is not portable ● Break = break compatibility

Slide 65

Slide 65 text

API Break header.h #include “header.h” int main() { void fly(10.0f); return 0; } mylib Version 1 library void fly(float speed); mylib Version 2 library void fly(float speed, float altitude); myprogram.c Execution environment Compiled with Version 1 myprogram.exe myprogam Compiled with Version 2 mylib.dll mylib.so

Slide 66

Slide 66 text

ABI Break ● C/C++ has the One Definition Rule ○ If two entity with same name but different definition >> Undefined behavior Execution environment Compiled with Version 1 myprogram.exe myprogam Compiled with Version 2 mylib.dll mylib.so mylib Version 1 library struct MyStruct{ int a; int b; }; mylib Version 2 library struct MyStruct{ int a; int b; Int c; }; header.h #include “header.h” int main() { MyStruct s; s.a = 1; s.b = 2; return 0; } myprogram.c

Slide 67

Slide 67 text

How to break ABI • Easy to break – Data layout change – Class structure change – Change method to virtual – … • References – KDE、Oracle、WG21

Slide 68

Slide 68 text

API and ABI Break in production: The SDKs ● Source code uses DCC tool SDKs ○ C/C++ Headers (Maya, 3DS Max, Nuke, Houdini, Arnold, AE, Photoshop) ○ Libraries ○ 3rd party libraries binaries ○ Documentation ■ API ■ Build instructions ● Plugins compatibility ○ Usually compatible within a major version ■ Maya, Nuke, Arnold, Nuke ■ ABI and API stable ■ Read the docs ○ Rarely compatible across major versions ■ After effects ● Depends on the feature set used ● API is based on suites and opaque types7

Slide 69

Slide 69 text

● C++ runtime /standard library break ● Plugins are executed on the client machine ● Plugins uses C/C++ >> C++ runtime library ● Windows: used to recompile until 2015 ○ Compatible dll VC2015 ■ Can mix VC++ versions libraries if link with most recent one ■ Exceptions for /GL and /LTCG (optimization options) ● Linux: libstdc++.so ○ About C++ ABI and the C++11 ABI break ○ Shipped with two version of the ABI ■ Switch using _GLIBCXX_USE_CXX11_ABI=0 or 1 API and ABI Break in production: C++ runtime VC++2017 error Error C2338… Before VS 2017 15.8, the member type would non-conformingly have an alignment of only alignof(max_align_t). VS 2017 15.8 was fixed to handle this correctly, but the fix inherently changes layout and breaks binary compatibility…

Slide 70

Slide 70 text

Libstdc++ ABI version ● C++11 String changed with GCC 5.1(2015) ○ Performance reasons (Copy on write) ○ ABI break ○ Can check ABI ■ objdump -T lib.so |grep cxx11 ■ nm-D lib.so |grep cxx11 ● VFX Reference platform ○ Switched in 2023 ○ Maya 2024 ○ Nuke 15 ○ Build with and without new ABI ■ We still use Maya 2023 ■ Arnold does not depend on C++ ● But MtoA does and load Arnold shaders ■ How to do it >> more details in the dependency section

Slide 71

Slide 71 text

Libstdc++ ABI version ● You have to use the right compiler ● Each DCC tools recommends a compiler ● Maya 2024 linux: DTS-11 + GCC 11.2.1 ○ Not GCC 11.2.1! DTS-11 (Development toolset) ● Maya 2024 recommended linux ○ Rocky Linux 8.7>> gcc 8.5 >> libstdc++ ABI GLIBCXX_3.4.25 ○ GCC 11.2.1 >> libstdc++ ABI GLIBCXX_3.4.29 ■ Can get version `GLIBCXX_3.4.26' not found error message ○ DTS-11 ■ Link to system library dynamically ■ Link statically for newer features ■ Uses GCC11 but libstdc++ ABI GLIBCXX_3.4.25 ■ DTS 6-7 ignore the ABI maccro

Slide 72

Slide 72 text

● Juts recompile everything ○ Most DCCs require plugins build for each SDK major release ■ Recompile against each SDK version ■ Use recommended compiler version ○ Two flavors on linux ■ _GLIBCXX_USE_CXX11_ABI=1 ■ _GLIBCXX_USE_CXX11_ABI=0 ■ Even for arnold plugins ● Loaded through MtoA ● ○ OS major update may mean need to recompile ● Third party libraries ○ Compile from sources ○ Need to propagate the ABI flavor recursively to dependencies ■ Use the right flavor during the build ● Don't expect ABI stability in general ○ Glibc had abi break ○ Deprecated attribute since c++14 Conclusion about API and ABI Break in production

Slide 73

Slide 73 text

Dev: Build About CMake

Slide 74

Slide 74 text

Build system ● Manually create build files ○ Make files ○ Visual studio ○ Ninja ● Generate the build system ○ Scons ○ Cmake ○ Cross platform ● CMake is a defacto standard ○ Used to generate Ninja, VC++… ○ Has to be Modern CMake Code Internal Library Code External Library Code Internal Library Binary External Library Binary Compiled Code Executable Code Compile Link Build System How to compile How to Link

Slide 75

Slide 75 text

Build definition with CMake Code Internal Library Code External Library Code Internal Library Binary External Library Binary Compiled Code Executable Code Compile Link … add_executable(application1) target_sources(application1 PRIVATE src.cpp src.h) target_compile_features(application1 PRIVATE cxx_std_14) find_package(library1) target_link_libraries(application1 PRIVATE library1)

Slide 76

Slide 76 text

Modern CMake ● So many ways to do Cmake ○ From cryptic one to modern cmake ● Modern CMake is about Best Practices ○ Use subset of CMake the right way ■ Mainly target based function ○ Use find_package for dependencies ○ De-facto standard ● Cmake documentation is great ○ Everything is there! Read carefully! ○ Reference ○ Mastering CMake ● Lots of references ○ Modern CMake for Modular Design (Cpp con, 2017) ○ Effective CMake (C++ now, 2017) ○ More Modern CMake (Meeting C++, 2018) ○ Embracing Modern CMake (NDC 2019) ○ Oh No! More Modern CMake (Meeting C++, 2019) ○ Modern CMake Modules (CppCon, 2021) ○ An Introduction to Modern Cmake (2023) ● Can generate the Visual Studio files for the developer ○ Guarantees binary consistency

Slide 77

Slide 77 text

Modern CMake in production ● Most of third party libraries have CMake support ○ OpenCV, Qhull, Eigen… ● Each DCC use specific compiler ○ Use Generator ○ Setting specific GCC is not simple when multiple GCC installed! ■ Prepend the PATH ■ Use scl ● DCC support of cmake is variable ○ Maya ■ Cmake but not modern ■ Not defined as targets ■ Impossible to use the Qt cmake out of the box ○ Nuke ■ Modern CMake with add_nuke_plugin wrapping ○ Houdini ■ Modern CMake ○ Arnold, After Effects, Photoshop ■ No cmake only Visual Studio file template ● Need CMake for Maya, AE, Arnold, Photoshop

Slide 78

Slide 78 text

Dev: Build Dependencies

Slide 79

Slide 79 text

Dependencies ● About libraries ● Resolution ● Diamond problem ● With Cmake ● With Package managers ● ABI issues ● In production ○ About SDKs

Slide 80

Slide 80 text

About libraries ● Why use libraries ○ Avoid to reinvent the wheel ○ Needed to use API ● Type of libraries ○ Dynamic or static ○ With or without source code

Slide 81

Slide 81 text

Dynamic/shared libraries ● Pro ○ Can be updated ■ If backward compatible ■ Enable to correct bug, security issues ○ Enable plugin system ■ All DCC plugins are dynamic libraries ● Cons ○ Update may cause problems ■ Change output ■ Introduce bugs ○ May be updated without your knowledge ■ OS update ■ Software install ○ More ABI problem ○ Need to deploy

Slide 82

Slide 82 text

Static libraries ● Pro ○ No need to deploy ○ Reduce conflicts for internal libraries ○ Reduce conflicts for third party libraries ■ May use third party library in their static fashion ○ Historical reasons from DLL hell on windows ● Cons ○ Usually not distributed so need to compile yourself ○ Large size binaries ■ Using Opencv_world from OpenCV in static mode results in huge binaries

Slide 83

Slide 83 text

Execution environment The dependencies resolution Code Library Binary Compiled Code Executable Code Library Header … Compile Required libraries Description Dependency resolution Dynamic Library Binary Library Binary Link

Slide 84

Slide 84 text

The dependencies resolution Compile Required libraries Description Required libraries Description Found binary? Found source? Library Binary Yes No Yes Library Code No Error Required libraries Description Dependency resolution Provides Execution environment Code Library Binary Compiled Code Executable Code Library Header … Compile Dynamic Library Binary Library Binary Link

Slide 85

Slide 85 text

● Two different paths of dependencies ● Same library ● Same version ● In CMake ○ config/module should make it ok ○ Library target defined only once ○ Safeguard with target existence check ■ Example in OpenCV config The diamond dependency problem: simple version Code Library A v1 Library B v1 Library C v1 Uses Uses Uses Uses if(NOT TARGET opencv_core) include(${CMAKE_CURRENT_LIST_DIR} /OpenCVModules ${OpenCV_MODULES_SUFFIX} .cmake) endif()

Slide 86

Slide 86 text

● Two different paths of dependencies ● Same library ● Different versions ● In Cmake ○ First call to find_package finds a compatible library ■ Use the config version file ○ Second call usually not change the targets ■ May depends on the config and version cmake ● Just DON'T ○ ABI issues ■ Get an object from Library C through Library A ■ Use the object with library B ■ Maybe result in undefined behavior ○ Find a common version compatible with all The diamond dependency problem: complex version Code Library A v1 Library B v1 Library C v1 Uses Uses Uses Uses Library C v2

Slide 87

Slide 87 text

The dependencies resolution in CMake Compile Required libraries Description Required libraries Description Found binary? Found source? Library Binary Yes No Yes Library Code No Error Required libraries Description Dependency resolution Provides Execution environment Code Library Binary Compiled Code Executable Code Library Header … Compile Dynamic Library Binary Library Binary Link FetchContent find_package

Slide 88

Slide 88 text

CMake find_package: how package are found? ● In the command line ○ Add each library CMake to the CMAKE_PREFIX_PATH and CMAKE_MODULE_PATH ○ Very long and tedious to type ■ Duplicate dependencies/ Need to know all of them ■ Ok for one project, not for automation ■ Rather use a package manager ● Module mode ○ In the CMAKE_MODULE_PATH ○ For libraries that don’t know CMake ○ For libraries that don’t have CMake config files ○ Usually written by the consumer of the library ○ Pre defined cmake modules ■ System ones and often installed ones ■ OpenMP, OpenGL, Git… ● Config mode ○ In the CMAKE_PREFIX_PATH ○ Recommended one ○ Provided by the library authors

Slide 89

Slide 89 text

CMake find_package: how package are found? ● Forward call to dependency provider ○ Can be a package manager ■ Vcpkg, Conan ■ Use chain "CMAKE_TOOLCHAIN_FILE" ■ Use CMake presets to make it simple ○ A custom dependency provider ● May use a CMake Package Registry ○ Local repository of libraries ○ Setup is OS dependent

Slide 90

Slide 90 text

CMake FetchContent ● Download a CMake project ● Compile the project as if it was part of the sources ● All project may not be FetchContent compatible ○ Same behavior if installed or built from source ● Download option if dependency not found through find_package ○ Added in the project find_package(googletest NAMES GTest) ) include(FetchContent) FetchContent_Declare( googletest GIT_REPOSITORY https://github.com/google/googletest.git GIT_TAG 703bd9caab50b139428cea1aaff9974ebee5742e # release-1.10.0 FIND_PACKAGE_ARGS NAMES GTest ) FetchContent_MakeAvailable(googletest) add_executable(ThingUnitTest thing_ut.cpp) target_link_libraries(ThingUnitTest GTest::gtest_main)

Slide 91

Slide 91 text

● Third party libraries should provide CMake ○ If installed ■ Add to CMAKE_PREFIX_PATH ■ Use find_package ○ Not installed ■ Use FetchContent ● Bring in the project and build from sources ● Equivalent to add_subdirectory ● Internal libraries ○ Process as third party ■ Create config package, Install, add to CMAKE_PREFIX_PATH ■ Use find_package ○ Build from source ■ Use FetchContent to download and include ■ Include in the project directly with add_subdirectory ● Need to be in the directory tree of the main CMakeList.txt ● Ideally want to only consume libraries through find_package!! CMake dependency resolution in practice

Slide 92

Slide 92 text

How to fallback to FetchContent ● Add download if necessary ○ Just wrap FetchContent ● Create a FindXXX.cmake for each library ○ Just call the wrapping function ● Recompile from source function (get_lib_from_github git_repo_name lib_name lib_version source_lib) set(git_url "https://github.com/${git_repo_name}.git ") #may need to add some authentication to the URL #set(git_url "https://x-access-token:${GITHUB_TOKEN}@github.com/ ${git_repo_name} .git") set(search_dir ${lib_name} /${lib_version} ) find_package (Git) if(IS_DIRECTORY "${search_dir}" ) #repo already downloaded once execute_process (COMMAND ${GIT_EXECUTABLE} pull ${git_url} ${lib_version} WORKING_DIRECTORY ${search_dir} ) set (${lib_name} _FOUND ON PARENT_SCOPE) set (${lib_name} _SEARCH_DIR ${search_dir} PARENT_SCOPE) if (source_lib) add_subdirectory (${search_dir} ${lib_name} _bin) endif () else () #try to download from git if (NOT TARGET ${lib_name} ) include (FetchContent) FetchContent_Declare( ${lib_name} GIT_REPOSITORY ${git_url} GIT_TAG ${lib_version} #OVERRIDE_FIND_PACKAGE SOURCE_DIR ${search_dir} ) FetchContent_MakeAvailable( ${lib_name} ) set (${lib_name} _FOUND ON PARENT_SCOPE) set (${lib_name} _SEARCH_DIR ${search_dir} PARENT_SCOPE) endif () endif () endfunction () Function to fetch from github

Slide 93

Slide 93 text

Simpler dependencies using package managers ● Vcpkg ○ Compile from source ○ Binary cache ○ Concept of baseline and triplet ■ Sure to always have the right library version? ■ Triplets are actually not triplets ○ Port concept may be complicated ○ Has a cmake toolchain ○ Need to duplicate dependencies in the manifest vcpkg.json if want specific version ■ Use override to get specific version ● Conan ○ May compile from source or download pre-build package ○ Duplicate dependencies in conanfile.txt or conanfile.py ○ Custom libraries/cache using artifactory ○ Has a cmake toolchain ● CPM ○ Setup free ○ Specific instruction CPMAddPackage ● CMake Package Registry ○ Just a binary cache to avoid recompilation ○ Need to call export(PACKAGE) from the project ■ Does not work from external libraries who don't support it

Slide 94

Slide 94 text

Simpler dependencies using package managers Compile Required libraries Description Required libraries Description Download binary? Download source? Library Binary Yes No Yes Library Code No Error Required libraries Description Dependency resolution Provides Conan In the cache? Yes No Execution environment Code Library Binary Compiled Code Executable Code Library Header … Compile Dynamic Library Binary Library Binary Link

Slide 95

Slide 95 text

Simpler dependencies using package managers Compile Required libraries Description Required libraries Description In the cache? Download source? Library Binary Yes No Yes Library Code No Error Required libraries Description Dependency resolution Provides Vcpkg Execution environment Code Library Binary Compiled Code Executable Code Library Header … Compile Dynamic Library Binary Library Binary Link

Slide 96

Slide 96 text

Simpler dependencies using package managers Compile Required libraries Description Source in the cache? Download source? Yes No Yes Library Code No Error Required libraries Description Dependency resolution CPM Execution environment Code Library Binary Compiled Code Executable Code Library Header … Compile Dynamic Library Binary Library Binary Link

Slide 97

Slide 97 text

Using package managers ● Plus ○ Easy to get library when change in library version ○ Easy to get library when change in compiler version ○ Use only find_package to get dependencies ■ CPM has specific instructions >> X ■ Wrap around FetchContent ● Minus ○ Requires setup (Vcpkg, Conan) ■ Any kind of automation would requires some setup ○ Requires a lot of compilation ■ Takes time ■ On each user machine ■ Using cache may help (Conan, Vcpkg, CMake) ● CMake Package Registry requires project to use export(PACKAGE) ● Complicated to share cache updates across machines ○ Dependencies duplication ■ Vcpkg ■ Conan

Slide 98

Slide 98 text

The simplest package managing ● External libraries manually compiled and stored in a local repo folder ○ May share archive among developer ○ May be on the company file servers ○ Common folder structure or specific environment variable name pointing to the repo ● Internal libraries have their own git repository ○ Use FetchContent wrap to download and compile ● CMakes utilities modules on a repo in the cloud ○ FindXXX.cmake files for each library ○ Each file just point to the directory in the repository ○ Can include utilities modules to manage ABI, FetchContent wraps… ● Add only the module folder to CMAKE_MODULE_PATH if (WIN32) set(OpenCV_DIR "${Path_to_OpenCV_Windows}")#windows, version else() set(OpenCV_DIR "${Path_to_OpenCV_Linux}" endif() find_package(OpenCV ${OpenCV_FIND_VERSION} CONFIG REQUIRED)#because OpenCv has config files Ex: FindOpenCV.cmake

Slide 99

Slide 99 text

Package managers and ABI ● Compile from source will use the global flag _GLIBCXX_USE_CXX11_ABI ● How to retrieve the correct ABI library? ○ Conan manages it in the profile? Just setting the compile? ■ Compiler.libcxx ○ Not sure how vcpkg manages it neither ■ Can compile, but how to retrieve ○ Find_package has no parameter to retrieve a library with a specific ABI ● Not sure ● Maybe use package manager in the future…

Slide 100

Slide 100 text

● Dependencies requirements are transitively applied by CMake B requires c++11 OpenMP Managing dependencies and ABI Library A Library B Program Uses Uses Union of A, B, C requirements C++17 OpenGL OpenMP A requires c++14 A requires from B OpenMP C requires C++17 OpenGL Library C Uses

Slide 101

Slide 101 text

● ABI has to be transmitted to dependencies Managing dependencies and ABI Library A Library B Program Uses with new ABI Requires New ABI Library C Uses with new ABI Uses with new ABI

Slide 102

Slide 102 text

How to get transitive new/old ABI at compile time? ● Define in the cmake command line! ○ New ABI: -D CMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=1' ○ Old ABI: -D CMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=0' ○ Compilation from source will be ok ● Package managers support ○ Conan supports it ■ Tools.gnu:define_libcxx11_abi ■ Is it transitive to dependencies? hopefully ○ Vcpkg compiles from sources ■ Need to define _GLIBCXX_USE_CXX11_ABI in VCPKG_CXX_FLAGS ■ Compile from sources, so should be transitive? ● However Cmake find_package does not support it ■ Need to ensure that find_package get the right pre-compiled dependencies!

Slide 103

Slide 103 text

Managing dependencies and ABI ● How to make find_package retrieve the correct ABI version? ○ Create/add to a module file ○ Add a test to the ABI flags, set the search directory accordingly ● Required for each library you use ○ Works well in conjunction with the simple package manager if (WIN32) set(OpenCV_DIR "${Path_to_OpenCV_Windows}" )#windows, version else() list(FIND CMAKE_CXX_FLAGS "-D_GLIBCXX_USE_CXX11_ABI=1" index_flags) if(index_flags GREATER_EQUAL 0) set(OpenCV_DIR "${Path_to_OpenCV_Linux_new_ABI}" else() set(OpenCV_DIR "${Path_to_OpenCV_Linux_old_ABI}" endif() endif() find_package (OpenCV ${OpenCV_FIND_VERSION} CONFIG REQUIRED) #because OpenCv has config files Ex: FindOpenCV.cmake Config mode avoids recursion Find the ABI global flag Point to the right folder according to the ABI flag

Slide 104

Slide 104 text

Dependencies in production ● All plugins are shared libraries ○ Shared libraries: need to deploy the dependencies with the plugin ○ Static libraries: easier deployment ● Need library exact version ○ SDKs are usually not compatible between versions ○ Guarantee image output ■ Semantic versioning does not take that into account ○ Use find_package( [version] EXACT) ■ Exact version is never guaranteed ■ Depends on the Module/Config processing of the parameters… ● How to handle DCC SDKs? ○ Package manager don't have DCC tool SDKs ■ Need to setup ○ Need a system to manage the SDKs

Slide 105

Slide 105 text

Dependencies in production: shared libraries ● Symbol conflicts ○ Different plugins, different version of the same library ■ Useful third party library like boost, opencv… ■ Internal libraries ● Plugins may have common dependencies with the host application ○ Maya lists ■ Boost, OpenimageIO, Ffmpeg, Qt… ○ Houdini lists ■ Boost, libtiff, zlib… ○ Nuke list ■ Jpeg, libpng ○ May result in diamond dependencies ● Vcpkg build shared libraries by default on Windows

Slide 106

Slide 106 text

How to manage SDKs ● SDKs are private ○ Cannot be put in public repositories in the cloud ■ Licences issues ■ Private repositories are ok ○ Cannot be downloaded directly from the web ■ Authentication process needed ● SDK may be huge ○ Maya SDK > 600Mb ■ X 8 (Windows, Linux, 3-4 Maya versions) = >4Gb ● Need to share across projects ○ Locally: machine of the programmer ■ Download an archive for developer to work locally ■ Download a repo from the cloud ■ May use LFS for binaries ■ Download limits (On Github: 2Gb, may use datapack) ■ File size limits (On Github: 100 Mb) ■ Repo size limits (On Github: around 5Gb?) ○ Remotely: use from file servers ■ Had issues on windows ● Too slow when using library files stored on linux servers ● Had to make a local copy…

Slide 107

Slide 107 text

How to manage SDKs with CMake ● Modern CMake with config file ○ Nuke ○ Houdini ● Cmake but no config file ○ Maya ● No Cmake ○ Arnold, After Effects, Photoshop ● Specific compilation steps ○ After effects and Photoshop uses PiPL resources files ○ Qt Moc files for Maya ■ Maya Qt is not useable out of the box ● Need FindXXX.cmake modules for the SDKs ○ Enable find_package use ● Need to wrap compilation specifics

Slide 108

Slide 108 text

● No lib, only headers ● Some common source files Example: After Effects set(AESDK_SEARCH_DIR ${EXT_LIB_FOLDER} /AESDK/Windows/ ${AESDK_FIND_VERSION} ) if (EXISTS "${AESDK_SEARCH_DIR}" ) message ("-- Found AESDK ${AESDK_FIND_VERSION}: ${AESDK_SEARCH_DIR}" ) set (AESDK_FOUND ON) if(NOT TARGET AESDK::headers ) add_library (AESDK::headers INTERFACE IMPORTED) target_include_directories (AESDK::headers INTERFACE "${AESDK_SEARCH_DIR}/Headers" "${AESDK_SEARCH_DIR}/Headers/SP" "${AESDK_SEARCH_DIR}/Headers/adobesdk" "${AESDK_SEARCH_DIR}/Util" ) target_compile_definitions (AESDK::headers INTERFACE _WINDOWS _WINDLL MSWindows) endif () if(NOT TARGET AESDK::source_utils) add_library (AESDK::source_utils INTERFACE IMPORTED) target_sources (AESDK::source_utils INTERFACE "${AESDK_SEARCH_DIR}/Util/AEGP_SuiteHandler.cpp" "${AESDK_SEARCH_DIR}/Util/MissingSuiteError.cpp" "${AESDK_SEARCH_DIR}/Util/Smart_Utils.cpp" "${AESDK_SEARCH_DIR}/Util/AEFX_SuiteHelper.c" ) target_include_directories (AESDK::source_utils INTERFACE "${AESDK_SEARCH_DIR}/Util" ) endif () else () message ("-- AESDK ${AESDK_FIND_VERSION} not found: ${AESDK_SEARCH_DIR}" ) set (AESDK_FOUND OFF) endif() FindAESDK.cmake Search in the lib repo Guard to avoid double inclusion Guard to avoid double inclusion Headers Common source code

Slide 109

Slide 109 text

Example: After Effects cmake_minimum_required (VERSION 3.20) project (MyPlugin LANGUAGES C CXX) add_library (aex_plugin SHARED) add_subdirectory (src) setup_pipl(aex_plugin src/pipl_file.r ${AE_VERSION} ) set_target_properties ( ${target_name} PROPERTIES SUFFIX .aex OUTPUT_NAME ${CMAKE_PROJECT_NAME} ) find_package (AESDK ${AESDK_VERSION} EXACT REQUIRED) target_link_libraries (${target_name} PRIVATE AESDK::headers) target_link_libraries (${target_name} PRIVATE AESDK::source_utils) install (TARGETS ${target_name} RUNTIME DESTINATION plugins/ ${AE_VERSION} /64/windows) set(out_dir ${CMAKE_BINARY_DIR} /$/ae) set_target_properties (${target_name} PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${out_dir} LIBRARY_OUTPUT_DIRECTORY ${out_dir} ARCHIVE_OUTPUT_DIRECTORY ${out_dir} Plugin sources PiPL processing Link to the SDK function (setup_pipl target_name pipl_file) target_sources (${target_name} PRIVATE ${pipl_file} ) add_custom_command (TARGET ${target_name} PRE_BUILD COMMAND cl.exe /D "${ae_macro}" /I "${GIT_LIB_FOLDER}/AESDK/${AESDK_VERSION}-windows/Headers" /I "${GIT_LIB_FOLDER}/AESDK/${AESDK_VERSION}-windows/Resources" /I "${CMAKE_CURRENT_SOURCE_DIR}/src" /EP ${pipl_file} > "${CMAKE_BINARY_DIR}/ae_effect_pipl.rr" COMMAND "${GIT_LIB_FOLDER}/AESDK/${AESDK_VERSION}-windows/Resources/PiPLtool.exe" "${CMAKE_BINARY_DIR}/ae_effect_pipl.rr" "${CMAKE_BINARY_DIR}/ae_effect_pipl.rrc" COMMAND cl.exe /D "MSWindows" /EP "${CMAKE_BINARY_DIR}/ae_effect_pipl.rrc" > "${CMAKE_BINARY_DIR}/ae_effect_pipl.rc" BYPRODUCTS "${CMAKE_BINARY_DIR}/ae_effect_pipl.rc" ) target_sources (${target_name} PRIVATE ${CMAKE_BINARY_DIR} /ae_effect_pipl.rc ) endfunction () Root CMakeFile.txt Function to process PiPL file SDK version is a variable

Slide 110

Slide 110 text

Summary about dependencies ● Use only find_package to get dependencies ○ Easier to switch to use a package manager later ● Avoid diamond dependencies ● Using package managers seems to be the future ○ Not sure about the C++11 ABI problem for now ○ Does not look very easy though ■ How to use Vcpkg post ■ First time i used VcPkg, i ran in an issue ● Dependencies folder is not that bad ● In production ○ Manage the SDKs ■ Create the cmake modules to find and configure them ○ Use static libraries is usually ok ● What about Git Submodules? ○ Seems hard to manage

Slide 111

Slide 111 text

Dev: Build CI

Slide 112

Slide 112 text

Automation and continuous integration ● Build automation ○ On each push ○ On various OS/Platform (Windows, Linux, iOS, Android, Game consoles…) ○ Manually trigger ● Use CI solution ○ Jenkins ○ Github actions ○ Gitlab CI ● Need script automation ○ Configuration of each repo with actions or CI/Jenkins task is cumbersome ○ Python

Slide 113

Slide 113 text

The production build matrix is huge ● Multiple OS support ○ Windows workstation for users ○ Mac for artists ○ Linux for render farm ● Multiple projects in parallel ○ Multiple tools version to support ○ Multiple DCC tools version to support ● Plugins ○ Target a specific SDK ○ Just like another OS support but… ■ Less stable than OS ■ More frequent breaking update ■ New version with new requirements ● OS ● Compiler/C++ standard version Development projects (excluding pipeline) DCC Active Support Inactive DCC Versio ns OS Maya 3 30 57 4 2 Arnold 3 18 8 4 2 AE 3 25 26 3 1 Nuke 1 6 13 2 2 Photoshop 1 3 - 2 1 Standalone 2 3 - 2 Tools: Active 13, Support 75, Targets: Active ~ 250 Support ~2000 New Year (each new dcc version): ~200 New Tool Version: ~15 Tools x Tool Versions x DDC versions x OS DCC Number of targets =

Slide 114

Slide 114 text

Jenkins ● Declarative pipeline ● Groovy script ○ Java based language ● Allow to launch compilation on various machines ○ Helps to automate linux/windows compilation ● Use python call from jenkins to automate tasks ○ Easy loop ○ Avoid to configure each and every tool in jenkins ○ Allow for easy and massive automation Jenkins CMake commands Python Execute Execute

Slide 115

Slide 115 text

Example of build Automation using Jenkins x Python ● Create task list from user input ○ Tool repository name ○ Branch name = tool version ○ DCC names and versions ○ “all” wildcard to compile available version/dccs ○ Deploy options ■ More on that in the Deploy section ● Launch compilations on relevant OS ○ Trigger the python build script ○ Trigger the python install and deploy ■ Linux compilation files ■ Windows compilation files ■ Be careful about permission when you copy them ● Cmake didn't seem to work as expected ○ Created directories didn't get the group permissions ● We copy linux files from windows

Slide 116

Slide 116 text

● Need tool listing file in readable/editable format ○ Json ○ Python ● Information about compilation ○ For each DCC + Version ■ Each OS >>> Compiler ■ Linux ● New C++11 ABI ● Old C++11 ABI ● Defined by DCC version ○ Specific info per DCC ■ Arnold <> Maya couples ■ Installed MtoA/Arnold list How to create the task list? maya_to_compiler_version = { "Windows": { "2019": "14.0", "2020": "14.1", "2022": "14.2", "2023": "14.2", "2024": "14.3", "2025": "14.3", }, "Linux": { "2019": "6.3.1", "2020": "6.3.1", "2022": "9.3.1", "2023": "9.3.1", "2024": "11.2.1", "2025": "11.2.1", } } nuke_to_compiler_version = { "Windows": { "13.2.3": "14.1", "14.1.2": "14.2", "15.0.1": "14.2"}, "Linux": { "13.2.3": "6.3.1", "14.1.2": "9.3.1", "15.0.1": "11.2.1"}, }

Slide 117

Slide 117 text

… "olm-toon ": { "tool_name ": "OLMToon ", "versions ": { "3.4": { "compile ": "scons", "dcc": {"arnold": {"dcc_versions ": config.arnold6 + config.arnold7, "sub_dir ": "maya"}, "maya": {"dcc_versions ": ["2022", "2023"]}, "mtoa": {"dcc_versions ": config.arnold6 + config.arnold7, "sub_dir ": "maya", }, }, }, … "4.2": { "compile ": "cmake", "dcc": { "arnold": {"dcc_versions ": config. versions_from ("arnold", "6.1.0.1 "),"sub_dir ": "maya"}, "maya": {"dcc_versions ": config. versions_from ("maya", "2022")}, "mtoa": {"dcc_versions ": config. versions_from ("arnold", "6.1.0.1 "),"sub_dir ": "maya"}, }, }, }, }, ● Tool listing contains ○ Repo name ○ Tool name ○ For each version ■ Supported DCC + versions How to create the task list?

Slide 118

Slide 118 text

● From tool listing, compiler and SDKs info generate the task list ● Each task has the following information ○ Repo name ○ Repo branch ○ OS to compile for ○ If linux use old c++ abi or new one ○ For deployment/file naming ■ Tool name ■ Install resources ○ Need to update the repository or not (run a clone or pull) How to create the task list?

Slide 119

Slide 119 text

Launch the tasks using Jenkins ● Automate what can be on Jenkins side ○ Trigger multiple python command line tool ● Be careful at what can be done sequentially and in parallel ● Clean in parallel ○ The previous build/installed files ● Update the repo in parallel ● Configure sequentially ○ Download of dependencies to the repo may result in disk writing collisions ● Build and Install in parallel ● Deploy in parallel ○ Only once all build are finished

Slide 120

Slide 120 text

Jenkins automation job_map_main = [:] //os:command:job_name:job job_map_deploy = [:] def add_job (job_map , os, command , job_name , job){ if (!job_map.containsKey(os)){ job_map[os] = [:] } if (!job_map[os].containsKey(command)){ job_map[os][command] = [:] } job_map[os][command][job_name]= job } def generate_stage (operation , repo_name , branch_name , dcc, dcc_version , install_resources , os, new_abi , olm_setup_install_dir , olm_setup_deploy_dir ) { return { stage( "${operation } ${repo_name } ${branch_name } ${dcc} ${dcc_version } ${install_resources } ${os} ${new_abi }") { script{ execute(operation, repo_name, branch_name, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir) } } } } def create_job (job_map , operation , repo_name , branch_name , dcc, dcc_version , install_resources , os, new_abi , olm_setup_install_dir , olm_setup_deploy_dir ){ job_id = repo_name + " " + operation + " " + branch_name + " " + dcc + " " + dcc_version + " "+install_resources+ " "+os + " " + new_abi add_job(job_map, os, operation, job_id, generate_stage(operation, repo_name, branch_name, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir)) } Generate closure to be evaluated later Store in maps

Slide 121

Slide 121 text

Jenkins automation pipeline { (…) stages { stage( 'Sequential ') { (…) script { def tool_def_list = get_compile_tasks(params.REPO_NAME, params.DCC_NAME, params.BRANCH_NAME, params.DCC_VERSION) for (d in tool_def_list){ (…) create_job(job_map_main, "clean",repo_name, version, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir) if (update_repository){ create_job(job_map_main, "update",repo_name, version, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir) } create_job(job_map_main, "configure ",repo_name, version, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir) create_job(job_map_main, "compile",repo_name, version, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir) if (os=="windows"){ create_job(job_map_main, "deploy",repo_name, version, dcc, dcc_version, install_resources, os, new_abi, olm_setup_install_dir, olm_setup_deploy_dir) } (…) } } Generate each job to execute with python

Slide 122

Slide 122 text

Jenkins automation (…) stage( 'Cleaning ') { parallel { stage( 'Clean Windows '){ agent {label "windows"} steps{ script { job_chunk_windows_clean = get_job_map(job_map_main, "windows", "clean") execute_job_parallel(job_chunk_windows_clean, get_chunk_size()) } } } stage( 'Clean Linux ') { agent {label "linux"} steps{ script { job_chunk_linux_clean = get_job_map(job_map_main, "linux", "clean") execute_job_parallel(job_chunk_linux_clean, get_chunk_size()) } } } } } (…) Clean on windows and linux in parallel Remap task to chunk Jenkins would have memory problem if not def execute_job_parallel(job_list, chunk_size){ chunk_list = map_to_list_chunk(job_list, chunk_size) for (chunk in chunk_list){ catchError() { parallel chunk } } }

Slide 123

Slide 123 text

Jenkins automation (…) stage( 'Launch jobs ') { parallel { stage( 'Launch windows cmake jobs ') { agent {label "windows"} steps { script { job_chunk_windows_update = get_job_map(job_map_main, "windows", "update") job_chunk_windows_configure = get_job_map(job_map_main, "windows", "configure ") job_chunk_windows_compile = get_job_map(job_map_main, "windows", "compile") execute_job_parallel(job_chunk_windows_update, get_chunk_size()) execute_job_serial(job_chunk_windows_configure) execute_job_parallel(job_chunk_windows_compile, get_chunk_size()) } } } stage( 'Launch linux cmake jobs ') { (…) } } } Launch the tasks in parallel on windows and linux Execute the python tasks Update in parallel Configure serial Compile in parallel We may run into Disk writing collision if we run two tasks at the same time with same tools Easiest work around is to disable run to instance of the pipeline concurrently

Slide 124

Slide 124 text

Python command line ● Low level command ● Execute only atomic part of the build ○ Clean|update|configure|compile|deploy python /cmake_main.py clean|update|configure|compile|deploy -root_folder -source_dir -install_dir -deploy_dir -repo_name -branch_name -dcc -dcc_version -libstd_new_abi -install_resources

Slide 125

Slide 125 text

Python command line ● Wraps Cmake call ● Sets the CMAKE_MODULE_PATH ● Sets the CMAKE_INSTALL_PREFIX ● Sets the generator ● Sets CMAKE_C_COMPILER, CMAKE_CXX_COMPILER ● Sets the "CMAKE_CXX_FLAGS='-D_GLIBCXX_USE_CXX11_ABI=1/0'", ● Sets DCC version macro since we compile for each dependency ○ ARNOLD_VERSION, MAYA_VERSION… ● Sets authentication token for Github ○ OLM_GITHUB_TOKEN ○ More details in the security part

Slide 126

Slide 126 text

About the build machines ● Use machine at your company is safer ○ Control on the configuration, libraries ○ Easier to debug build ● Build in the cloud ○ Legal issues ■ Maya: 2.1.1 No License Granted; Unauthorized Activities. ..(d) for Installation of the Licensed Materials on any Computer other than a Computer owned or leased, and controlled, by Licensee, unless otherwise authorized in writing by Autodesk ■ AE ● in the code :Dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Adobe Systems Incorporated. ● Developers are free to distribute plugins built using SDK headers and source files; you just can't distribute our SDK headers and source files. ■ Houdini ● In the code This software is proprietary to Side Effects Software Inc., and is not to be reproduced, transmitted, or disclosed in any way without written permission. ○ Money issues ■ Image available ● Split compile per compiler? ■ Cost for data transfer? Not sure how much. ● Dependencies are multiple Gb ○ Maya is 1GB! OpenCV is multiple Gb. Qt? Opencv? Boost? Eigen? ● Action cache 10 Gb limit, link to a single repo ■ Cost for github hosted

Slide 127

Slide 127 text

Summary of build for production ● Maybe in some years we don't care about ABI problem anymore? ○ Deprecated attribute since c++14 ● Hopefully we use package manager for everything ○ Custom dependency provider for the SDKs? ● Massive automation is needed ○ Now using Jenkins ○ May move to github self hosted runner ■ More secure ○ Need script based automation ● Still need to studies the compilation in the cloud ○ Self hosted runners seems to be fine but slow

Slide 128

Slide 128 text

Dev: test

Slide 129

Slide 129 text

Test ● Ensure software quality ● Test ○ Unit Tests ○ Integration tests ○ User tests ● Better to have tests automated ● Test libraries ○ Doctest ○ Catch ○ Google Test ○ Typical use ■ Use your API and the test API to write test program ■ Build test as independent executable ■ Run the tests at/after build

Slide 130

Slide 130 text

Test in production ● Plugins run within host applications ○ Unit testing is challenging ○ Need a lot of user input and context ■ Could use mock? ● Movies and cartoon primary quality is the picture! ○ Between two versions of a tool ■ Is output the same? ■ Is time the same? ● Test the image output ○ All DCCs output images ○ Need to run renders and compare them ○ 3D tools ■ Modeling/Animation/Rigging: same output >> same shape >> same render ■ Shading/Lighting: same output >> same color >> same render ○ 2D tools ■ Drawing: same strokes >> same render ■ Compositing: same picture generation/processing >> same render

Slide 131

Slide 131 text

How to test rendering output ● Create the reference ○ Choose tool version reference ○ Choose DCC version reference ○ Create reference DCC file ■ Using the tool to test ■ Use and animate all parameters ■ Keep the number of frame to render low ○ Render the reference ● Create the variations ○ Variation on DCC version ○ Variation on tool version ○ Render the variation ● Compare the reference and the variations ● Requirements ○ Use lossless image format ○ Program to compare images (count different pixels, use metrics…) ○ Need automation of rendering (command line, render farm…)

Slide 132

Slide 132 text

How to use results and iterate Test Name DCC/Tool VS Reference Comparison to Reference Meaning S/S Same Normal Different Should not happen. Error in your reference or in your test process S/D Same Tool version stable on this DCC version Update the reference if D/S fails. Different Unexpected: check you code. Expected: update the reference. D/S Same DCC output stable. Can try D/D test to check tool stability. Different DCC output unstable. Update reference or create multiple references. D/D Same DCC and tool stable. No need to update the reference. Different DCC or tool is unstable. Run D/S to check. S: same D: Different

Slide 133

Slide 133 text

Example of a test framework in After Effects ● What we want to test ○ Tool features ■ Vary parameter, animate… ○ AE feature support ■ Render in 8/16/32 bits ■ Mask ■ Downsampling, non square pixels ■ Time remap ■ Applied after gaussian blur ● Create a test composition ○ Use a specific version of the tool ○ Generate multiple composition to cover the AE feature to tests ● Render using output modules ○ Use Tiff with lzw compression (works well for anime pictures) ○ Bit settings also set by output module ○ Create 8/16/32 bit output modules, install to rendering machine ■ Quite annoying step… ● Compare using libtiff custom compare tool ○ Imagemagick was too slow

Slide 134

Slide 134 text

Conclusion about test ● Needed ● When to update the reference may be tricky ○ Make tools to update references when needed ● Automation of tests run with build ● Each DCC may requires some specific adjustments ○ Photoshop does not have command line rendering capabilities ○ AE parameters cannot all be animated and changed by scripts ● How to test UI interaction? ○ By hand? …

Slide 135

Slide 135 text

Ops in production

Slide 136

Slide 136 text

Overview ● Installing software and plugins ● Release ○ The development environment ● Deploy ○ The test and production environment ● Operate ● Monitor

Slide 137

Slide 137 text

Installing software and plugins ● Software install ○ Use an installer ■ Content and installer need to be signed ○ Multiple machines need command line installer + automation script ● Plugin installation ○ Depend on the host applications ■ Copy in a directory ■ Set environment variables

Slide 138

Slide 138 text

Installing plugins in production ● Maya ○ Modules ■ .mod file to describe environment variables, platforms, local, module version ■ Can set every other environment variables ○ Environment variables ■ Plugins: MAYA_PLUG_IN_PATH ● Need to get your own Maya node IDs ■ Mel Script: MAYA_SCRIPT_PATH ■ Python script: PYTHONPATH ■ Bifrost:BIFROST_LIB_CONFIG_FILES for components and plugins ■ Icons: XBMLANGPATH ■ Presets:MAYA_PRESET_PATH ○ Maya.env: set of maya environment variables, don't override system ones ■ One for each maya version in user home directory ○ userSetup.mel, userSetup.py: script executed at startup of the application ● Arnold ○ Nodes/Procedurals/Shaders: ARNOLD_PLUGIN_PATH ○ Python templates: MTOA_TEMPLATES_PATH ● MtoA ○ Translators: MTOA_EXTENSIONS_PATH

Slide 139

Slide 139 text

Installing plugins in production ● AE ○ Plugins: copy to the plugin folder ○ Scripts: load from the script folder ● Photoshop ○ CEP extensions: copy to the extension folder ○ Plugins: copy to the plugin folder ● Nuke ○ Plugins and Gizmo: NUKE_PATH ○ OpenFX plugins: OFX_PLUGIN_PATH ○ Python scripts: PYTHONPATH ● Houdini ○ Various environment variables (HOUDINI_PATH, HOUDINI_DSO_PATH, HOUDINI_MENU_PATH…)

Slide 140

Slide 140 text

Release ● Copy the plugins locally ○ File will be locked at DCC tool startup ● Update environment variables if necessary ● Launch the application ● Everything should be automated

Slide 141

Slide 141 text

Release is more complicated ● Multiple project in parallel ○ Platform (DCC tool versions) set at the start of the project ○ Similar to video games ● Multiple tool versions X multiple DCC tool version ● Application launcher ○ Input ■ Tool list and version for each project as input ■ Repository: dev, test, prod ○ Automate the setup of the DCC Tool ● Need 3 different release environments ○ For developers to test >> Dev ○ For artists to test >> Test ○ For artist to use in production >> Production

Slide 142

Slide 142 text

Example of application launcher ● Command line mode for the render farm ● UI mode for the users (...) Application to launch Vendor plugins process as other tools

Slide 143

Slide 143 text

Release and Deploy ● Release ○ Use in real conditions ○ Quality assurance tests ○ Prepare to send to final user ○ In production ■ Release = install the plugins to the development repository ● Deploy ○ Send to final user ■ Installer, download ■ Web service update ○ In production ■ Release = install the plugins to the test/production repository

Slide 144

Slide 144 text

Deploy and versions in production ● Develop from last release in production to the next one ● Increment version compared to last one ○ May change during development according to version semantic ○ Freeze version after release to production Tool x.y.z Tool x.y.z Tool u.v.w Tool x.y.z With x = u+1 OR y=v+1 OR z=w+1 Production to Development Test to Production Test to Development Development to test

Slide 145

Slide 145 text

Operate in production ● The system guys side ○ Client machine ○ Servers ○ Network ○ Software install and licences ● The pipeline side ○ Assets management ○ Rendering wrangler ○ Pipeline tools… ● Everyone cares about the artist feedback ○ Tracking and manage ○ Using tools described in the plan phase

Slide 146

Slide 146 text

What could be monitor in production ● Crash report and logs ● How the tool is used? ● How often the tool is used? ● How fast is the rendering? ● Who uses the tool? ● Can derives metrics ○ Tool popularity and impact ○ Identify bottlenecks ● Be aware ○ Some tools are essentially used at specific phases ■ Ex: rigging tool only used at rig stage ○ Some tools are used by very specialized people ■ Ex: hair grooming tools

Slide 147

Slide 147 text

Security

Slide 148

Slide 148 text

Code Plan Build Test Release Deploy Operate Monitor DEV OPS DCC licenses Binary/Installer sign CI solution (Jenkins, Github Actions) SCM Servers, Machines, Licences Trello, Jira, Figma Google Apps VC++ DCC licenses SCM Developer account (VC++, Qt) OS Account SCM (Github, Gitlab) Remote connection (Splashtop, VPN, SSH) Many logins and authentication

Slide 149

Slide 149 text

About security ● Login in various account and authentication ○ User to machine ■ On site machine ■ Remotely ● From home ● At the company (unifies the way to access pc) ○ User to web service ■ SCM in the cloud (Github, Gitlab…) ■ Google apps ○ Service to service ■ Github <> Google Cloud ■ Jenkins <> Github ■ Github self hosted Runner <> Github ● Installer on windows ○ Requires binaries and DLLs to be signed ○ CPack is not enough

Slide 150

Slide 150 text

Many technologies… ● Old world ○ VPN ○ SSH ● New world ○ HTTPS ○ Two step authentication ○ Security Assertion Markup Language (SAML) ○ Single sign on ○ Open iD Connect (OiDC) ■ Workload Identity (Google) ■ AWS, Azure

Slide 151

Slide 151 text

Example: SAML ● User logins using identity provider ○ Windows account, google account ● When access to any service provider SAML triggered ○ Check with identity provider ○ If ok grants access with a token ● Benefits ○ No need for the user to sign in for each service ○ Used by single sign on

Slide 152

Slide 152 text

Principles ● Add security on the authentication ○ Connect identity provider to service provider ○ Identity provider emit a token ■ Limited permissions/credentials ■ Limited in time ■ Limit damages in case of break ○ Service provider trusts identity provider ● Simplify user experience ○ Less password to remember or change

Slide 153

Slide 153 text

Service to service authentication ● Should use the same principles as user to service ● OiDC to connect Identity provider to Service provider ○ On top of OAuth2.0 ○ Uses HTTPS ● HTTPS ○ Need to get a SSL certificate from a certificate authority (ex: GlobalSign) ○ Need to manage certificate (they expire) ○ Use port 80, 443 ■ Usually open, useable in most environment ○ Recommended by github ● Identity provider using OiDC ○ Google workload identity ○ AWS, Azure ● Service provider able to use OiDC ○ Github ○ Gitlab ○ Google, AWS, Azure ● Identity provider manage the time limited tokens for you

Slide 154

Slide 154 text

Service to service authentication in production ● Lots of automations ● Authentication with SCM in the cloud (Github, Gitlab…) ○ Compile machine ○ CI solutions ● Sign of binaries and installers ○ Requires access to certificate stored in a secure place ■ With authentication ■ Ex: Google drive ● Ideally: implement OiDC ● Sometimes ○ Use PAT, github app authentication, … ○ SSH

Slide 155

Slide 155 text

Authentication at OLM ● Google Account ○ Authentication: Single Sign On + SAML ○ Two step authentication ● Splashtop ○ With single sign on (from google account) ○ Connect to PC through Splashtop even at the company ■ Less latency than home ● Jenkins ○ Internal server ○ Our instance does not support HTTPS ■ Cannot connect to Workload identity ● Github ○ With single sign on (from google account) ○ Two step authentication ■ Mandatory since 2023 ○ Https for users ○ CI solution connects through Github apps ■ Cannot use a specific user account ● not the right way to do it, what about the password? ■ Generate temporary token use in every git commands ■ Generate a pem file ● Does not expire! Like ssh private key ● Better than PAT but not ideal ■ Plan to move to Github self hosted runners + Github actions + Google Workload Identity

Slide 156

Slide 156 text

Not recommended solutions ● VPN ○ Security: attack target ○ Update requires network stop ■ No update >> attack ○ Balance between connection speed and cost ● SSH ○ No screen, slow ■ Remote desktop ■ VNC ○ Need to open 22 port ○ Private key ■ Per user ■ User can generate and create ■ Use password manager ■ Leak of private key problem ■ Maybe no time limits ● Personal Access Token PAT ○ Couldn't use Fine-grained PAT in the past ○ Need to decide how much right to give, the user can decide ○ Related to one person: what when the persons leaves the company?

Slide 157

Slide 157 text

● C++ modules ○ Need more experience with it ○ Has to flag the module files (Compiler, cmake…) ● SVN, Mercurial, Scons ○ Not used that much ones ● Github actions ○ Working on it to replace Jenkins ● Gitlab ○ CI seems to be popular and very well designed ● Ninja ○ Seem to speed up compilation time ● Rust ○ Some effort to use Rust in CG existed but Still alive? ● Qt ○ de facto standard for UI. Moc generation may need some specific management ● Javascript ○ Specific versions in After effects, Photoshop ● Python ○ Used in every major DCC package ○ Switch from python 2 to 3 was quite painful What we didn't talk about: techs ● MacOS ○ Used by a lot on artists. We can requests to support macos for our open tools ● Conan ○ Don't know the share of vcpkg VS Conan, Seem to have more flexibility than Vcpkg, but seem more complex ● Jira ○ Integrates tickets, reports and docs together. Good to manage the feedback and specifications? Tried some years ago… ● SPK and SPFS ○ For complex management of packages in CG pipelines ● Rez ○ Prepare environment to launch application. Used in production ● Docker ○ latform for developing, shipping, and running applications ● Other DCCs ○ Houdini, Cinema4D, 3dsMax, Katana, Blender… cannot cover all of them. Maybe in the future.

Slide 158

Slide 158 text

What we didn't talk about: methods ● Waterfall VS Agile: we think agile is more appropriate for production ● Specifications ○ So many levels of abstraction ■ UI concepts ■ Functionality overviews ■ … ■ UML diagrams ■ C++ function signature + expected output ○ How to write? ■ Most docs i found are either too high or too low level ■ In the end … ad hoc ■ For UI ● Figma ● Google slides

Slide 159

Slide 159 text

Conclusion ● Plan ○ Manage the backlog ○ Be sure to keep communication going on with artists ● Code ○ Rules are important ○ Specific semantic versioning for production ● Build ○ Many build to achieve ○ Automation is mandatory ○ ABI!! ● Test ○ Testing code in a host application is complicated ○ Testing picture is different than testing code ● Release/Deploy ○ Application launcher is mandatory ● Operate ○ Be sure that the artists can focus on art ○ Always listen to what they have to say ● Monitor ○ Automation of report and logging ○ Use logs for future improvements

Slide 160

Slide 160 text

DevOps Journey at OLM

Slide 161

Slide 161 text

Evolution of DevOps in 15 years at OLM Digital 15 years ago Now WIP Staff 2 developers, 1 researcher 10+ developers, 1 researcher Issue tracking Nothing Trello Trello + Git projects Source code management Nothing SVN/Git Git Package manager Nothing In-house In-House (Git, CMake) Build Visual studio VS, GCC, Scons, Jenkins VS, GCC, Cmake, Github actions Deployment Manual copy /installers (wix) to clients Automatic copy to render servers Automatic DCC tool configuration Automatic install to clients/servers Continuous integration Nothing Jenkins (legacy jobs) Jenkins (pipeline scripts) Integration tests for CG Tool versioning Nothing Semantic versioning for CG tool

Slide 162

Slide 162 text

Why changing technology? ● Move from SVN to Git ○ Central server on site at OLM ○ Access from outside with splashtop ■ Latency problems with remote work ● Foreign internship student ● Foreign employees working from outside of Japan ○ Up to 550 ms latency using simple web page test ● Working from different parts in Japan ○ Collaboration with other companies ● Move from scons to cmake ○ Our Scons solution was not implemented correctly ■ Needed rework ○ Cmake became a standard in the meantime ○ Cmake affinity with git is great

Slide 163

Slide 163 text

Future works at OLM ● Plan/Operate ○ Connect report/feedback with coding: Jira? ○ How to write spec? ● Build ○ Self hosted runner with Github actions ○ Use package manager? vcpkg? ● Test ○ Testing using user simulation through system like AutoHotKey ● Release/Deploy ○ More automation to manage the project tool needs ○ Installer through Self hosted runner with Github actions ● Monitor ○ Actually monitor and generate some useful statistics

Slide 164

Slide 164 text

Future works for this course ● Other techs? ○ Gitlab, Jira. Rez, Conan, Docker… ● Other DCCs? ○ Houdini, Katana… ● How to switch from SVN to Git? ● How to switch from Scons to Cmake? ● More CMake ○ Details about each DCCs CMakes? ○ Dependencies? ○ Link to package managers? ● Github actions ● More case studies? ● Who wanna collaborate for a 3 hours DevOps for production course?

Slide 165

Slide 165 text

Code Plan Build Test Release Deploy Operate Monitor DEV OPS Release to test environment Release to production environment Automatic test after build Servers, Machines, Licences Messages, Automatic/Manual data collection Getting the artist in the room Survey,Track issues Multiple DCCs, DCCs versions, Multiple OS Multiple host languages for plugins

Slide 166

Slide 166 text

Reference about Agile ● Scrum is like a war room situation ● https://devops.com/agile-scrum-is-a-failure-heres-why/ ● https://stackoverflow.blog/2020/06/29/does-scrum-ruin-great-engineers-or-are -you-doing-it-wrong/ ● https://techbeacon.com/app-dev-testing/why-scrum-sucks ● https://bootcamp.uxdesign.cc/scrum-is-just-as-bad-as-safe-ac4f1d0f1193 ● https://blog.logrocket.com/why-scrum-is-becoming-irrelevant/ ● https://www.linkedin.com/pulse/why-does-scrum-make-programmers-hate-co ding-jayme-edwards/ ● https://www.linkedin.com/pulse/kanban-considered-harmful-chris-nicola/ ● https://www.lean.org/the-lean-post/articles/how-can-kanban-be-useful-in-softw are-since-we-never-produce-the-same-part-twice/