Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using transtats to track translation string change

Using transtats to track translation string change

Presented in Flock (the Fedora Project's annual contributor-focused conference) 2018 at The Radisson Blu Park Hotel & Conference Centre, Dresden Radebeul, Germany.


Sundeep Anand

August 09, 2018


  1. Using transtats to track translation string change Flock 2018 Sundeep

    Anand (suanand) Parag Nemade (pnemade)
  2. Topics Why transtats • Challenges • Touchpoints How to use

    jobs • Server • Client What’s next • Developments • Integrations Get Involved • Try & test • Contribute
  3. Why transtats?

  4. Challenges • How to track... ◦ Strings of packages have

    been duly pushed/pulled to/from respective Translation Platform? ◦ Latest strings are being packaged and build in koji? • What is the volume of translation required for next fedora release? • Can we fix major translation bugs reported after translation deadline, before final freeze? • Which set of languages need attention? • Is there any way to automate few steps? • By comparing translation statistics derived from different places: upstream repository, translation platform and build system. • Transtats release summary can tell us volume of translation required per languages (summing up statistics of all packages). • YAML based job can parse packaged SRPM to check criticality of translation bugs. • Transtats can answer - which packages are out of sync (has diff) and in which languages? • By adding a few more tasks to YAML, automation can be achieved.
  5. Touchpoints Software String Freeze Translation Deadline string change coverage translation

    update volume Out of Sync Packages languages need attention
  6. How to use Jobs?

  7. Transtats Jobs Create Run Save Schedule Share ... Predefined Jobs

    • Sync Build Tags • Sync Translation Platform • Sync Release Calendars YML Based Jobs • Sync Upstream • Sync Downstream • String Change • Verify Translations Inspect Jobs • List View • Detail View
  8. Sync YML Job Templates job: exception: raise execution: sequential name:

    upstream stats package: %PACKAGE_NAME% return_type: json tasks: - clone: git repo - filter: PO files - calculate: Stats type: syncupstream job: buildsys: %BUILD_SYSTEM% exception: raise execution: sequential name: downstream stats package: %PACKAGE_NAME% return_type: json tags: %BUILD_TAG% tasks: - get: latest build info - download: SRPM - unpack: SRPM - load: Spec file - unpack: tarball - apply: patch - filter: PO files - calculate: Stats type: syncdownstream
  9. Track String Change YML Job Templates job: exception: raise execution:

    sequential name: string change package: %PACKAGE_NAME% release: %RELEASE_SLUG% return_type: json tasks: - clone: - name: git repo - branch: master - generate: - name: POT file - domain: %PACKAGE_NAME% - cmd: cd po && intltool-update --pot --gettext-package=%PACKAGE_NAME% - download: Platform POT file - calculate: Diff type: stringchange Under Development - verifytrans Steps: • Download and unpack latest SRPM from build system. • Create fresh translation template (POT file) from source tarball. • Pull translations from translation platform. • Merge pulled translations to newly created POT – set 1 • Apply patch and filter PO files – set 2 • Compare set 1 vs set 2 – find diff for each locale
  10. if diff: # string change is detected if after_string_freeze_date: #

    it should be a breakage! send_warning_to_maintainer() if after_translation_deadline: # may be an error send_alert_to_quality_engineer()
  11. Just four steps Pick Template Define Tasks Set Values Trigger

  12. Transtats Demo Server Side - transtats.xyz Youtube https://youtu.be/D1sHpxfxrk4

  13. Transtats CLI

  14. Transtats APIs 1. Ping Server: Returns server version GET <transtats_server>/api/ping

    2. Package Status: Returns translation stats of package for enabled languages, for example abrt. GET <transtats_server>/api/package/<package_name> 3. Graph Rule Coverage: Returns translation coverage according to graph rule, for example rhinstaller. GET <transtats_server>/api/coverage/<graph_rule_name> 4. Release Status: Returns translation stats of packages which are being tracked for a given release, for example fedora-29. GET <transtats_server>/api/release/<release_branch_name> 5. Job Details: Returns job log against given job id, for example 2a6d4b23-6a6b-4d0e-b617-a0ece01d790f. GET <transtats_server>/api/job/<job-id>/log
  15. Transtats CLI • Written in python • Uses REST API

    • Code hosted on github • Provides output in json format as well as normal text format cli now supports 5 commands • coverage • job • package • release • version
  16. Transtats CLI Demo Client Side - commands

  17. What’s next?

  18. Development Plans 1. Transtats deployment in fedora infrastructure. 2. More

    templates in YML based Jobs like verifytrans etc. 3. Better UI, to eliminate navigation confusions. (New Dashboard) 4. Transtats jobs scheduling feature. To automate. 5. Notifications about string breakage, translation deadlines etc. 6. Better user guide. Improve docs.transtats.org
  19. Transtats UI Demo New Dashboard

  20. Integrations 1. Automated test plugin in update systems like bodhi

    2. Message Queues like fedmsg 3. Chat process like IRC bot.
  21. Get Involved ...

  22. Try & test: deployments Get docker daemon running. Build or

    pull transtats image (docker.io) and get started. Clone the repo and build the image $ git clone https://github.com/transtats/transtats.git $ cd transtats; sudo docker build -t transtats/transtats deploy/docker Or, Pull the image $ sudo docker pull docker.io/transtats/transtats Run the image $ sudo docker run -d --name container_name -p 8080:8015 transtats/transtats Application should be accessible at localhost:8080 Openshift • Install PostgreSQL Server >= 9.5 and make it accessible over FQDN. • Install minishift • Modify the database-host in secret.yml which should point to PostgreSQL host. • Run the following commands after you login to your cluster on terminal $ oc new-project transtats-deployment $ oc create -f deploy/openshift $ oc start-build transtats-build Once build/deploy finishes, application shall be accessible.
  23. Lets Make Transtats Better Together... $ git clone transtats ...

    $ sudo vagrant up $ sudo vagrant ssh $ cd /workspace $ make run http://localhost:8080 Transtats Server • A simple django application • Uses requests to talk to several services • Few APIs written in Django REST Framework • Forms are in django-crispy-forms • Uses PostgreSQL database • Uses fixture to load test data Transtats CLI • Uses click framework • Consume Server APIs • Few commands Transtats Automation • Selenium Project Transtats Ansible • Ansible playbooks Project: http://transtats.org Docs: http://docs.transtats.org Source: https://github.com/transtats
  24. Thank You #fedora-g11n #transtats feedback.transtats.xyz