Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How GitHub Supports Vim License Detection, The Five Years Journey

C4ce16f549c450f4759eb37f5d5d1a63?s=47 othree
July 30, 2022

How GitHub Supports Vim License Detection, The Five Years Journey

C4ce16f549c450f4759eb37f5d5d1a63?s=128

othree

July 30, 2022
Tweet

More Decks by othree

Other Decks in Programming

Transcript

  1. othree at COSCUP 2022 How GitHub Supports Vim License Detection

    The Five Years Journey
  2. othree at COSCUP 2022 GitHub ⽀援 Vim License 的故事 歷時五年的開源貢獻

  3. Who is othree • Web Developer • Mozillian (MozTW) •

    Blogger https://blog.othree.net • Speaker https://speakerdeck.com/othree • Vimmer • ❤ OSS
  4. Before We Start • GitHub and Open Source License •

    Vim License
  5. GitHub and Open Source License

  6. None
  7. None
  8. None
  9. License detection • Highlight selected License • Search by license

    • Only major licenses are supported • Detected by Licensee, a Ruby Gem
  10. The Vim License

  11. The Vim License • A special License only for Vim

    • GPL compatible according to Richard Stallman • Coupled with the Vim • Many Vim script uses Vim license, a.k.a “Same as Vim” • https://vimhelp.org/uganda.txt.html
  12. VIM LICENSE I) There are no restrictions on distributing unmodified

    copies of Vim except that they must include this license text. You can also distribute unmodified parts of Vim, likewise unrestricted except that they must include this license text. You are also allowed to include executables that you made from the unmodified Vim sources, plus your own usage examples and Vim scripts. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the
  13. VIM LICENSE I) There are no restrictions on distributing unmodified

    copies of Vim except that they must include this license text. You can also distribute unmodified parts of Vim, likewise unrestricted except that they must include this license text. You are also allowed to include executables that you made from the unmodified Vim sources, plus your own usage examples and Vim scripts. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the
  14. II) It is allowed to distribute a modified (or extended)

    version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the maintainer will do with your changes and under what license they will be distributed is negotiable. If there has been no negotiation then this license, or a later version, also applies to your changes. The current maintainer is Bram Moolenaar <Bram@vim.org>. If this changes it will be announced in appropriate places (most likely vim.sf.net, www.vim.org and/or comp.editors). When it is completely impossible to contact the maintainer, the obligation to send him your changes ceases. Once the maintainer has confirmed that he has received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it
  15. a) If you make changes to Vim yourself, you must

    clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the maintainer will do with your changes and under what license they will be distributed is negotiable. If there has been no negotiation then this license, or a later version, also applies to your changes. The current maintainer is Bram Moolenaar <Bram@vim.org>. If this changes it will be announced in appropriate places (most likely vim.sf.net, www.vim.org and/or comp.editors). When it is completely impossible to contact the maintainer, the obligation to send him your changes ceases. Once the maintainer has confirmed that he has received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it unmodified, as mentioned at I). If you make additional changes the text under a) applies to those changes. c) Provide all the changes, including source code, with every copy of the modified Vim you distribute. This may be done in the form of a context diff. You can choose what license to use for new code you
  16. received your changes they will not have to be sent

    again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it unmodified, as mentioned at I). If you make additional changes the text under a) applies to those changes. c) Provide all the changes, including source code, with every copy of the modified Vim you distribute. This may be done in the form of a context diff. You can choose what license to use for new code you add. The changes and their license must not restrict others from making their own changes to the official version of Vim. d) When you have a modified Vim which includes changes as mentioned under c), you can distribute it without the source code for the changes if the following three conditions are met: - The license that applies to the changes permits you to distribute the changes to the Vim maintainer without fee or restriction, and permits the Vim maintainer to include the changes in the official version of Vim without fee or restriction. - You keep the changes for at least three years after last distributing the corresponding modified Vim. When the maintainer or someone who you distributed the modified Vim to asks you (in any way) for the changes within this period, you must make them available to him.
  17. - You clearly describe in the distribution how to contact

    you. This contact information must remain valid for at least three years after last distributing the corresponding modified Vim, or as long as possible. e) When the GNU General Public License (GPL) applies to the changes, you can distribute the modified Vim under the GNU GPL version 2 or any later version. 3) A message must be added, at least in the output of the ":version" command and in the intro screen, such that the user of the modified Vim is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is
  18. - You clearly describe in the distribution how to contact

    you. This contact information must remain valid for at least three years after last distributing the corresponding modified Vim, or as long as possible. e) When the GNU General Public License (GPL) applies to the changes, you can distribute the modified Vim under the GNU GPL version 2 or any later version. 3) A message must be added, at least in the output of the ":version" command and in the intro screen, such that the user of the modified Vim is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is
  19. is able to see that it was modified. When distributing

    as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is <maintainer@vim.org> IV) It is not allowed to remove this license from the distribution of the Vim sources, parts of it or from a modified version. You may use this license for previous Vim releases instead of the license that they came with, at your option.
  20. None
  21. The Vim License • A lot of word Vim in

    the text • The Vim Maintainer
  22. VIM LICENSE I) There are no restrictions on distributing unmodified

    copies of Vim except that they must include this license text. You can also distribute unmodified parts of Vim, likewise unrestricted except that they must include this license text. You are also allowed to include executables that you made from the unmodified Vim sources, plus your own usage examples and Vim scripts. II) It is allowed to distribute a modified (or extended) version of Vim, including executables and/or source code, when the following four conditions are met: 1) This license text must be included unmodified. 2) The modified Vim must be distributed in one of the following five ways: a) If you make changes to Vim yourself, you must clearly describe in the distribution how to contact you. When the maintainer asks you (in any way) for a copy of the modified Vim you distributed, you must make your changes, including source code, available to the maintainer without fee. The maintainer reserves the right to include your changes in the official version of Vim. What the maintainer will do with your changes and under what license they will be distributed is negotiable. If there has been no negotiation then this license, or a later version, also applies to your changes. The current maintainer is Bram Moolenaar <Bram@vim.org>. If this changes it will be announced in appropriate places (most likely vim.sf.net, www.vim.org and/or comp.editors). When it is completely impossible to contact the maintainer, the obligation to send him your changes ceases. Once the maintainer has confirmed that he has received your changes they will not have to be sent again. b) If you have received a modified Vim that was distributed as mentioned under a) you are allowed to further distribute it unmodified, as mentioned at I). If you make additional changes the text under a) applies to those changes. c) Provide all the changes, including source code, with every copy of the modified Vim you distribute. This may be done in the form of a context diff. You can choose what license to use for new code you add. The changes and their license must not restrict others from making their own changes to the official version of Vim. d) When you have a modified Vim which includes changes as mentioned under c), you can distribute it without the source code for the changes if the following three conditions are met: - The license that applies to the changes permits you to distribute the changes to the Vim maintainer without fee or restriction, and permits the Vim maintainer to include the changes in the official version of Vim without fee or restriction. - You keep the changes for at least three years after last distributing the corresponding modified Vim. When the maintainer or someone who you distributed the modified Vim to asks you (in any way) for the changes within this period, you must make them available to him. - You clearly describe in the distribution how to contact you. This contact information must remain valid for at least three years after last distributing the corresponding modified Vim, or as long as possible. e) When the GNU General Public License (GPL) applies to the changes, you can distribute the modified Vim under the GNU GPL version 2 or any later version. 3) A message must be added, at least in the output of the ":version" command and in the intro screen, such that the user of the modified Vim is able to see that it was modified. When distributing as mentioned under 2)e) adding the message is only required for as far as this does not conflict with the license used for the changes. 4) The contact information as required under 2)a) and 2)d) must not be removed or changed, except that the person himself can make corrections. III) If you distribute a modified version of Vim, you are encouraged to use the Vim license for your changes and make them available to the maintainer, including the source code. The preferred way to do this is by e-mail or by uploading the files to a server and e-mailing the URL. If the number of changes is small (e.g., a modified Makefile) e-mailing a context diff will do. The e-mail address to be used is <maintainer@vim.org> IV) It is not allowed to remove this license from the distribution of the Vim sources, parts of it or from a modified version. You may use this license for previous Vim releases instead of the license that they came with, at your option.
  23. Motivation

  24. Guide to Contribution

  25. None
  26. None
  27. None
  28. 2017

  29. First Issue

  30. None
  31. None
  32. None
  33. None
  34. 2018

  35. None
  36. None
  37. None
  38. 2019

  39. Contribute Again

  40. None
  41. Requirements 1. The license must have an SPDX identifier If

    your license isn't registered with SPDX, please request that it be added 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search
  42. 1. The license must have an SPDX identifier If your

    license isn't registered with SPDX, please request that it be added 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search
  43. 1. The license must have an SPDX identifier If your

    license isn't registered with SPDX, please request that it be added 2. The license must be listed on one of the following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search 4. 3 notable projects using the license must be identified. These
  44. 2. The license must be listed on one of the

    following approved lists of licenses: * List of OSI approved licenses * GNU's list of free licenses 
 (*note: the license must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search 4. 3 notable projects using the license must be identified. These must have straightforward LICENSE files which serve as examples newcomers can follow and that could be detected by licensee if it knew about the license.
  45. * GNU's list of free licenses 
 (*note: the license

    must be listed in one of the three "free" categories*) * Open Definition's list of conformant licenses 3. The license must be used in at least *1,000* public repositories. This may be documented, for example, with a GitHub code search 4. 3 notable projects using the license must be identified. These must have straightforward LICENSE files which serve as examples newcomers can follow and that could be detected by licensee if it knew about the license.
  46. None
  47. None
  48. None
  49. None
  50. 3 Notable Projects • Vim • pathogen.vim by Tim Pope

    • TBD
  51. None
  52. None
  53. None
  54. None
  55. Substitutions

  56. Substitutions Vim → software? package? project? Vim Maintainer → softwareMaintainer?

    …etc
  57. Should I?

  58. None
  59. Yes, But , But

  60. Not The Vim Maintainer

  61. Everything is Ready

  62. None
  63. The Licensee

  64. Licensee • A Ruby Gem to detect under what license

    a project is distributed • Use data from choosealicense.com • Only detects picked licenses • Supports LICENSE, package fi le, statements in README, SPDX fi le …
  65. The matching method • Exact match(hash) • Sørensen–Dice coe ff

    i cient (aka Dice) • Similarity of two sets(word set) • String length
  66. None
  67. The Issue • There are too many [project] in the

    license text • After replaced all substitutions to Vim. The con fi dence < 98
  68. Possible solutions • Add tolerance to the speci fi ed

    license • Use regex if there are substitutions • Dynamic tolerance based on the substitutions
  69. None
  70. None
  71. None
  72. SPDX License List

  73. Software Package Data Exchange (SPDX) • Open standard for communicating

    software bill of material information • From Linux Foundation • An ISO standard (ISO/IEC 5962:2021) • Example
  74. SPDX License List • Most popular standard by SPDX •

    Standardize license full name, text, identi fi er • A canonical permanent URL for each license • Supported by most eco system(ex: npm, composer, cabal)
  75. None
  76. None
  77. None
  78. First Pull Request

  79. None
  80. None
  81. None
  82. None
  83. The Result • Mike Linksvayer already create a PR to

    fi x the issue • Add extra tolerance to substitution fi elds
  84. None
  85. vim-license.dev

  86. Vim License Gen • A web app help you generate

    Vim License Text fi le with custom project name
  87. None
  88. None
  89. 2020

  90. The 2nd/3rd Pull Request

  91. • Add LICENSE fi le to the notable projects •

    vim • pathogen.vim
  92. None
  93. The Main Pull Request

  94. 3 notable projects • Vim • pathogen.vim by Tim Pope

    • vim-license-gen (aka vim-license.dev)
  95. None
  96. Merged !!

  97. None
  98. None
  99. Wait for Licensee Update

  100. None
  101. Wait for Licensee Release

  102. None
  103. Wait for GitHub to Upgrade

  104. None
  105. Wait and Wait • Give zero pressure to the GitHub

    • Wait and wait • Then one day, I test the Licensee again
  106. None
  107. None
  108. The Cause • The change of the similarity algorithm •

    No test case for Vim License
  109. Fix Licensee

  110. The Solution • Update the algorithm or manual decision parameter

    • Modi fi ed a parameter from 10 to 13 • Add test case for Vim License
  111. None
  112. The Result • Mike Linksvayer already create a PR to

    fi x the issue • For each substitution fi eld, provide 2 ch space not counting to the penalty • Merged at Oct 2020
  113. None
  114. Wait Again

  115. None
  116. Feb 2021

  117. GitHub Support

  118. None
  119. None
  120. Wait Again and Again

  121. None
  122. Jan 2022

  123. GitHub Search by License

  124. None
  125. Vim’s License Still Not Correct

  126. None
  127. None
  128. GitHub Support

  129. None
  130. None
  131. None
  132. None
  133. None
  134. Summary

  135. • 2 issues • 5 pull requests • 1 mailing-list

    thread • 1 new web app • 2 GitHub support request • ~ 5 years
  136. Take away

  137. • How to start a CONTRIBUTION and how to communicate

    • What is SPDX and SPDX License List • What is Licensee and how it work • The deep understand of Vim License
  138. Thanks for You Listening

  139. Questions?