Using transtats to track translation string change

Presented in Flock (the Fedora Project's annual contributor-focused conference) 2018 at The Radisson Blu Park Hotel & Conference Centre, Dresden Radebeul, Germany.

Transcript

1. Using transtats to track translation string change Flock 2018 Sundeep Anand (suanand) Parag Nemade (pnemade)
2. Topics Why transtats • Challenges • Touchpoints How to use jobs • Server • Client What’s next • Developments • Integrations Get Involved • Try & test • Contribute
3. Why transtats?
4. Challenges • How to track... ◦ Strings of packages have been duly pushed/pulled to/from respective Translation Platform? ◦ Latest strings are being packaged and build in koji? • What is the volume of translation required for next fedora release? • Can we fix major translation bugs reported after translation deadline, before final freeze? • Which set of languages need attention? • Is there any way to automate few steps? • By comparing translation statistics derived from different places: upstream repository, translation platform and build system. • Transtats release summary can tell us volume of translation required per languages (summing up statistics of all packages). • YAML based job can parse packaged SRPM to check criticality of translation bugs. • Transtats can answer - which packages are out of sync (has diff) and in which languages? • By adding a few more tasks to YAML, automation can be achieved.
5. Touchpoints Software String Freeze Translation Deadline string change coverage translation update volume Out of Sync Packages languages need attention
6. How to use Jobs?
7. Transtats Jobs Create Run Save Schedule Share ... Predefined Jobs • Sync Build Tags • Sync Translation Platform • Sync Release Calendars YML Based Jobs • Sync Upstream • Sync Downstream • String Change • Verify Translations Inspect Jobs • List View • Detail View
8. Sync YML Job Templates job: exception: raise execution: sequential name: upstream stats package: %PACKAGE_NAME% return_type: json tasks: - clone: git repo - filter: PO files - calculate: Stats type: syncupstream job: buildsys: %BUILD_SYSTEM% exception: raise execution: sequential name: downstream stats package: %PACKAGE_NAME% return_type: json tags: %BUILD_TAG% tasks: - get: latest build info - download: SRPM - unpack: SRPM - load: Spec file - unpack: tarball - apply: patch - filter: PO files - calculate: Stats type: syncdownstream
9. Track String Change YML Job Templates job: exception: raise execution: sequential name: string change package: %PACKAGE_NAME% release: %RELEASE_SLUG% return_type: json tasks: - clone: - name: git repo - branch: master - generate: - name: POT file - domain: %PACKAGE_NAME% - cmd: cd po && intltool-update --pot --gettext-package=%PACKAGE_NAME% - download: Platform POT file - calculate: Diff type: stringchange Under Development - verifytrans Steps: • Download and unpack latest SRPM from build system. • Create fresh translation template (POT file) from source tarball. • Pull translations from translation platform. • Merge pulled translations to newly created POT – set 1 • Apply patch and filter PO files – set 2 • Compare set 1 vs set 2 – find diff for each locale
10. if diff: # string change is detected if after_string_freeze_date: # it should be a breakage! send_warning_to_maintainer() if after_translation_deadline: # may be an error send_alert_to_quality_engineer()
11. Just four steps Pick Template Define Tasks Set Values Trigger Job
12. Transtats Demo Server Side - transtats.xyz Youtube https://youtu.be/D1sHpxfxrk4
13. Transtats CLI
14. Transtats APIs 1. Ping Server: Returns server version GET <transtats_server>/api/ping 2. Package Status: Returns translation stats of package for enabled languages, for example abrt. GET <transtats_server>/api/package/<package_name> 3. Graph Rule Coverage: Returns translation coverage according to graph rule, for example rhinstaller. GET <transtats_server>/api/coverage/<graph_rule_name> 4. Release Status: Returns translation stats of packages which are being tracked for a given release, for example fedora-29. GET <transtats_server>/api/release/<release_branch_name> 5. Job Details: Returns job log against given job id, for example 2a6d4b23-6a6b-4d0e-b617-a0ece01d790f. GET <transtats_server>/api/job/<job-id>/log
15. Transtats CLI • Written in python • Uses REST API • Code hosted on github • Provides output in json format as well as normal text format cli now supports 5 commands • coverage • job • package • release • version
16. Transtats CLI Demo Client Side - commands
17. What’s next?
18. Development Plans 1. Transtats deployment in fedora infrastructure. 2. More templates in YML based Jobs like verifytrans etc. 3. Better UI, to eliminate navigation confusions. (New Dashboard) 4. Transtats jobs scheduling feature. To automate. 5. Notifications about string breakage, translation deadlines etc. 6. Better user guide. Improve docs.transtats.org
19. Transtats UI Demo New Dashboard
20. Integrations 1. Automated test plugin in update systems like bodhi 2. Message Queues like fedmsg 3. Chat process like IRC bot.
21. Get Involved ...
22. Try & test: deployments Get docker daemon running. Build or pull transtats image (docker.io) and get started. Clone the repo and build the image $ git clone https://github.com/transtats/transtats.git $ cd transtats; sudo docker build -t transtats/transtats deploy/docker Or, Pull the image $ sudo docker pull docker.io/transtats/transtats Run the image $ sudo docker run -d --name container_name -p 8080:8015 transtats/transtats Application should be accessible at localhost:8080 Openshift • Install PostgreSQL Server >= 9.5 and make it accessible over FQDN. • Install minishift • Modify the database-host in secret.yml which should point to PostgreSQL host. • Run the following commands after you login to your cluster on terminal $ oc new-project transtats-deployment $ oc create -f deploy/openshift $ oc start-build transtats-build Once build/deploy finishes, application shall be accessible.
23. Lets Make Transtats Better Together... $ git clone transtats ... $ sudo vagrant up $ sudo vagrant ssh $ cd /workspace $ make run http://localhost:8080 Transtats Server • A simple django application • Uses requests to talk to several services • Few APIs written in Django REST Framework • Forms are in django-crispy-forms • Uses PostgreSQL database • Uses fixture to load test data Transtats CLI • Uses click framework • Consume Server APIs • Few commands Transtats Automation • Selenium Project Transtats Ansible • Ansible playbooks Project: http://transtats.org Docs: http://docs.transtats.org Source: https://github.com/transtats
24. Thank You #fedora-g11n #transtats feedback.transtats.xyz