Slide 1
Slide 1 text
othree at COSCUP 2022
How GitHub Supports Vim License Detection
The Five Years Journey
Slide 2
Slide 2 text
othree at COSCUP 2022
GitHub ⽀援 Vim License 的故事
歷時五年的開源貢獻
Slide 3
Slide 3 text
Who is othree
• Web Developer
• Mozillian (MozTW)
• Blogger https://blog.othree.net
• Speaker https://speakerdeck.com/othree
• Vimmer
• ❤ OSS
Slide 4
Slide 4 text
Before We Start
• GitHub and Open Source License
• Vim License
Slide 5
Slide 5 text
GitHub and Open Source License
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
License detection
• Highlight selected License
• Search by license
• Only major licenses are supported
• Detected by Licensee, a Ruby Gem
Slide 10
Slide 10 text
The Vim License
Slide 11
Slide 11 text
The Vim License
• A special License only for Vim
• GPL compatible according to Richard Stallman
• Coupled with the Vim
• Many Vim script uses Vim license, a.k.a “Same as Vim”
• https://vimhelp.org/uganda.txt.html
Slide 12
Slide 12 text
VIM LICENSE
I) There are no restrictions on distributing unmodified copies of Vim except
that they must include this license text. You can also distribute
unmodified parts of Vim, likewise unrestricted except that they must
include this license text. You are also allowed to include executables
that you made from the unmodified Vim sources, plus your own usage
examples and Vim scripts.
II) It is allowed to distribute a modified (or extended) version of Vim,
including executables and/or source code, when the following four
conditions are met:
1) This license text must be included unmodified.
2) The modified Vim must be distributed in one of the following five ways:
a) If you make changes to Vim yourself, you must clearly describe in
the distribution how to contact you. When the maintainer asks you
(in any way) for a copy of the modified Vim you distributed, you
must make your changes, including source code, available to the
maintainer without fee. The maintainer reserves the right to
include your changes in the official version of Vim. What the
Slide 13
Slide 13 text
VIM LICENSE
I) There are no restrictions on distributing unmodified copies of Vim except
that they must include this license text. You can also distribute
unmodified parts of Vim, likewise unrestricted except that they must
include this license text. You are also allowed to include executables
that you made from the unmodified Vim sources, plus your own usage
examples and Vim scripts.
II) It is allowed to distribute a modified (or extended) version of Vim,
including executables and/or source code, when the following four
conditions are met:
1) This license text must be included unmodified.
2) The modified Vim must be distributed in one of the following five ways:
a) If you make changes to Vim yourself, you must clearly describe in
the distribution how to contact you. When the maintainer asks you
(in any way) for a copy of the modified Vim you distributed, you
must make your changes, including source code, available to the
maintainer without fee. The maintainer reserves the right to
include your changes in the official version of Vim. What the
Slide 14
Slide 14 text
II) It is allowed to distribute a modified (or extended) version of Vim,
including executables and/or source code, when the following four
conditions are met:
1) This license text must be included unmodified.
2) The modified Vim must be distributed in one of the following five ways:
a) If you make changes to Vim yourself, you must clearly describe in
the distribution how to contact you. When the maintainer asks you
(in any way) for a copy of the modified Vim you distributed, you
must make your changes, including source code, available to the
maintainer without fee. The maintainer reserves the right to
include your changes in the official version of Vim. What the
maintainer will do with your changes and under what license they
will be distributed is negotiable. If there has been no negotiation
then this license, or a later version, also applies to your changes.
The current maintainer is Bram Moolenaar . If this
changes it will be announced in appropriate places (most likely
vim.sf.net, www.vim.org and/or comp.editors). When it is completely
impossible to contact the maintainer, the obligation to send him
your changes ceases. Once the maintainer has confirmed that he has
received your changes they will not have to be sent again.
b) If you have received a modified Vim that was distributed as
mentioned under a) you are allowed to further distribute it
Slide 15
Slide 15 text
a) If you make changes to Vim yourself, you must clearly describe in
the distribution how to contact you. When the maintainer asks you
(in any way) for a copy of the modified Vim you distributed, you
must make your changes, including source code, available to the
maintainer without fee. The maintainer reserves the right to
include your changes in the official version of Vim. What the
maintainer will do with your changes and under what license they
will be distributed is negotiable. If there has been no negotiation
then this license, or a later version, also applies to your changes.
The current maintainer is Bram Moolenaar . If this
changes it will be announced in appropriate places (most likely
vim.sf.net, www.vim.org and/or comp.editors). When it is completely
impossible to contact the maintainer, the obligation to send him
your changes ceases. Once the maintainer has confirmed that he has
received your changes they will not have to be sent again.
b) If you have received a modified Vim that was distributed as
mentioned under a) you are allowed to further distribute it
unmodified, as mentioned at I). If you make additional changes the
text under a) applies to those changes.
c) Provide all the changes, including source code, with every copy of
the modified Vim you distribute. This may be done in the form of a
context diff. You can choose what license to use for new code you
Slide 16
Slide 16 text
received your changes they will not have to be sent again.
b) If you have received a modified Vim that was distributed as
mentioned under a) you are allowed to further distribute it
unmodified, as mentioned at I). If you make additional changes the
text under a) applies to those changes.
c) Provide all the changes, including source code, with every copy of
the modified Vim you distribute. This may be done in the form of a
context diff. You can choose what license to use for new code you
add. The changes and their license must not restrict others from
making their own changes to the official version of Vim.
d) When you have a modified Vim which includes changes as mentioned
under c), you can distribute it without the source code for the
changes if the following three conditions are met:
- The license that applies to the changes permits you to distribute
the changes to the Vim maintainer without fee or restriction, and
permits the Vim maintainer to include the changes in the official
version of Vim without fee or restriction.
- You keep the changes for at least three years after last
distributing the corresponding modified Vim. When the maintainer
or someone who you distributed the modified Vim to asks you (in
any way) for the changes within this period, you must make them
available to him.
Slide 17
Slide 17 text
- You clearly describe in the distribution how to contact you. This
contact information must remain valid for at least three years
after last distributing the corresponding modified Vim, or as long
as possible.
e) When the GNU General Public License (GPL) applies to the changes,
you can distribute the modified Vim under the GNU GPL version 2 or
any later version.
3) A message must be added, at least in the output of the ":version"
command and in the intro screen, such that the user of the modified Vim
is able to see that it was modified. When distributing as mentioned
under 2)e) adding the message is only required for as far as this does
not conflict with the license used for the changes.
4) The contact information as required under 2)a) and 2)d) must not be
removed or changed, except that the person himself can make
corrections.
III) If you distribute a modified version of Vim, you are encouraged to use
the Vim license for your changes and make them available to the
maintainer, including the source code. The preferred way to do this is
by e-mail or by uploading the files to a server and e-mailing the URL.
If the number of changes is small (e.g., a modified Makefile) e-mailing a
context diff will do. The e-mail address to be used is
Slide 18
Slide 18 text
- You clearly describe in the distribution how to contact you. This
contact information must remain valid for at least three years
after last distributing the corresponding modified Vim, or as long
as possible.
e) When the GNU General Public License (GPL) applies to the changes,
you can distribute the modified Vim under the GNU GPL version 2 or
any later version.
3) A message must be added, at least in the output of the ":version"
command and in the intro screen, such that the user of the modified Vim
is able to see that it was modified. When distributing as mentioned
under 2)e) adding the message is only required for as far as this does
not conflict with the license used for the changes.
4) The contact information as required under 2)a) and 2)d) must not be
removed or changed, except that the person himself can make
corrections.
III) If you distribute a modified version of Vim, you are encouraged to use
the Vim license for your changes and make them available to the
maintainer, including the source code. The preferred way to do this is
by e-mail or by uploading the files to a server and e-mailing the URL.
If the number of changes is small (e.g., a modified Makefile) e-mailing a
context diff will do. The e-mail address to be used is
Slide 19
Slide 19 text
is able to see that it was modified. When distributing as mentioned
under 2)e) adding the message is only required for as far as this does
not conflict with the license used for the changes.
4) The contact information as required under 2)a) and 2)d) must not be
removed or changed, except that the person himself can make
corrections.
III) If you distribute a modified version of Vim, you are encouraged to use
the Vim license for your changes and make them available to the
maintainer, including the source code. The preferred way to do this is
by e-mail or by uploading the files to a server and e-mailing the URL.
If the number of changes is small (e.g., a modified Makefile) e-mailing a
context diff will do. The e-mail address to be used is
IV) It is not allowed to remove this license from the distribution of the Vim
sources, parts of it or from a modified version. You may use this
license for previous Vim releases instead of the license that they came
with, at your option.
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
The Vim License
• A lot of word Vim in the text
• The Vim Maintainer
Slide 22
Slide 22 text
VIM LICENSE
I) There are no restrictions on distributing unmodified copies of Vim except
that they must include this license text. You can also distribute
unmodified parts of Vim, likewise unrestricted except that they must
include this license text. You are also allowed to include executables
that you made from the unmodified Vim sources, plus your own usage
examples and Vim scripts.
II) It is allowed to distribute a modified (or extended) version of Vim,
including executables and/or source code, when the following four
conditions are met:
1) This license text must be included unmodified.
2) The modified Vim must be distributed in one of the following five ways:
a) If you make changes to Vim yourself, you must clearly describe in
the distribution how to contact you. When the maintainer asks you
(in any way) for a copy of the modified Vim you distributed, you
must make your changes, including source code, available to the
maintainer without fee. The maintainer reserves the right to
include your changes in the official version of Vim. What the
maintainer will do with your changes and under what license they
will be distributed is negotiable. If there has been no negotiation
then this license, or a later version, also applies to your changes.
The current maintainer is Bram Moolenaar . If this
changes it will be announced in appropriate places (most likely
vim.sf.net, www.vim.org and/or comp.editors). When it is completely
impossible to contact the maintainer, the obligation to send him
your changes ceases. Once the maintainer has confirmed that he has
received your changes they will not have to be sent again.
b) If you have received a modified Vim that was distributed as
mentioned under a) you are allowed to further distribute it
unmodified, as mentioned at I). If you make additional changes the
text under a) applies to those changes.
c) Provide all the changes, including source code, with every copy of
the modified Vim you distribute. This may be done in the form of a
context diff. You can choose what license to use for new code you
add. The changes and their license must not restrict others from
making their own changes to the official version of Vim.
d) When you have a modified Vim which includes changes as mentioned
under c), you can distribute it without the source code for the
changes if the following three conditions are met:
- The license that applies to the changes permits you to distribute
the changes to the Vim maintainer without fee or restriction, and
permits the Vim maintainer to include the changes in the official
version of Vim without fee or restriction.
- You keep the changes for at least three years after last
distributing the corresponding modified Vim. When the maintainer
or someone who you distributed the modified Vim to asks you (in
any way) for the changes within this period, you must make them
available to him.
- You clearly describe in the distribution how to contact you. This
contact information must remain valid for at least three years
after last distributing the corresponding modified Vim, or as long
as possible.
e) When the GNU General Public License (GPL) applies to the changes,
you can distribute the modified Vim under the GNU GPL version 2 or
any later version.
3) A message must be added, at least in the output of the ":version"
command and in the intro screen, such that the user of the modified Vim
is able to see that it was modified. When distributing as mentioned
under 2)e) adding the message is only required for as far as this does
not conflict with the license used for the changes.
4) The contact information as required under 2)a) and 2)d) must not be
removed or changed, except that the person himself can make
corrections.
III) If you distribute a modified version of Vim, you are encouraged to use
the Vim license for your changes and make them available to the
maintainer, including the source code. The preferred way to do this is
by e-mail or by uploading the files to a server and e-mailing the URL.
If the number of changes is small (e.g., a modified Makefile) e-mailing a
context diff will do. The e-mail address to be used is
IV) It is not allowed to remove this license from the distribution of the Vim
sources, parts of it or from a modified version. You may use this
license for previous Vim releases instead of the license that they came
with, at your option.
Slide 23
Slide 23 text
Motivation
Slide 24
Slide 24 text
Guide to Contribution
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
2017
Slide 29
Slide 29 text
First Issue
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
2018
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
2019
Slide 39
Slide 39 text
Contribute Again
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
Requirements
1. The license must have an SPDX identifier
If your license isn't registered with SPDX, please request that it be added
2. The license must be listed on one of the following approved
lists of licenses:
* List of OSI approved licenses
* GNU's list of free licenses
(*note: the license must be listed in one of the three
"free" categories*)
* Open Definition's list of conformant licenses
3. The license must be used in at least *1,000* public repositories.
This may be documented, for example, with a GitHub code search
Slide 42
Slide 42 text
1. The license must have an SPDX identifier
If your license isn't registered with SPDX, please request that it be added
2. The license must be listed on one of the following approved
lists of licenses:
* List of OSI approved licenses
* GNU's list of free licenses
(*note: the license must be listed in one of the three
"free" categories*)
* Open Definition's list of conformant licenses
3. The license must be used in at least *1,000* public repositories.
This may be documented, for example, with a GitHub code search
Slide 43
Slide 43 text
1. The license must have an SPDX identifier
If your license isn't registered with SPDX, please request that it be added
2. The license must be listed on one of the following approved
lists of licenses:
* List of OSI approved licenses
* GNU's list of free licenses
(*note: the license must be listed in one of the three
"free" categories*)
* Open Definition's list of conformant licenses
3. The license must be used in at least *1,000* public repositories.
This may be documented, for example, with a GitHub code search
4. 3 notable projects using the license must be identified. These
Slide 44
Slide 44 text
2. The license must be listed on one of the following approved
lists of licenses:
* List of OSI approved licenses
* GNU's list of free licenses
(*note: the license must be listed in one of the three
"free" categories*)
* Open Definition's list of conformant licenses
3. The license must be used in at least *1,000* public repositories.
This may be documented, for example, with a GitHub code search
4. 3 notable projects using the license must be identified. These
must have straightforward LICENSE files which serve as examples
newcomers can follow and that could be detected by licensee
if it knew about the license.
Slide 45
Slide 45 text
* GNU's list of free licenses
(*note: the license must be listed in one of the three
"free" categories*)
* Open Definition's list of conformant licenses
3. The license must be used in at least *1,000* public repositories.
This may be documented, for example, with a GitHub code search
4. 3 notable projects using the license must be identified. These
must have straightforward LICENSE files which serve as examples
newcomers can follow and that could be detected by licensee
if it knew about the license.
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
No content
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
3 Notable Projects
• Vim
• pathogen.vim by Tim Pope
• TBD
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
No content
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
Substitutions
Slide 56
Slide 56 text
Substitutions
Vim →
software?
package?
project?
Vim Maintainer →
softwareMaintainer?
…etc
Slide 57
Slide 57 text
Should I?
Slide 58
Slide 58 text
No content
Slide 59
Slide 59 text
Yes, But
, But
Slide 60
Slide 60 text
Not The Vim Maintainer
Slide 61
Slide 61 text
Everything is Ready
Slide 62
Slide 62 text
No content
Slide 63
Slide 63 text
The Licensee
Slide 64
Slide 64 text
Licensee
• A Ruby Gem to detect under what license a project is distributed
• Use data from choosealicense.com
• Only detects picked licenses
• Supports LICENSE, package
fi
le, statements in README, SPDX
fi
le …
Slide 65
Slide 65 text
The matching method
• Exact match(hash)
• Sørensen–Dice coe
ff i
cient (aka Dice)
• Similarity of two sets(word set)
• String length
Slide 66
Slide 66 text
No content
Slide 67
Slide 67 text
The Issue
• There are too many [project] in the license text
• After replaced all substitutions to Vim. The con
fi
dence < 98
Slide 68
Slide 68 text
Possible solutions
• Add tolerance to the speci
fi
ed license
• Use regex if there are substitutions
• Dynamic tolerance based on the substitutions
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
No content
Slide 72
Slide 72 text
SPDX License List
Slide 73
Slide 73 text
Software Package Data Exchange (SPDX)
• Open standard for communicating software bill of material information
• From Linux Foundation
• An ISO standard (ISO/IEC 5962:2021)
• Example
Slide 74
Slide 74 text
SPDX License List
• Most popular standard by SPDX
• Standardize license full name, text, identi
fi
er
• A canonical permanent URL for each license
• Supported by most eco system(ex: npm, composer, cabal)
Slide 75
Slide 75 text
No content
Slide 76
Slide 76 text
No content
Slide 77
Slide 77 text
No content
Slide 78
Slide 78 text
First Pull Request
Slide 79
Slide 79 text
No content
Slide 80
Slide 80 text
No content
Slide 81
Slide 81 text
No content
Slide 82
Slide 82 text
No content
Slide 83
Slide 83 text
The Result
• Mike Linksvayer already create a PR to
fi
x the issue
• Add extra tolerance to substitution
fi
elds
Slide 84
Slide 84 text
No content
Slide 85
Slide 85 text
vim-license.dev
Slide 86
Slide 86 text
Vim License Gen
• A web app help you generate Vim License Text
fi
le with custom project name
Slide 87
Slide 87 text
No content
Slide 88
Slide 88 text
No content
Slide 89
Slide 89 text
2020
Slide 90
Slide 90 text
The 2nd/3rd Pull Request
Slide 91
Slide 91 text
• Add LICENSE
fi
le to the notable projects
• vim
• pathogen.vim
Slide 92
Slide 92 text
No content
Slide 93
Slide 93 text
The Main Pull Request
Slide 94
Slide 94 text
3 notable projects
• Vim
• pathogen.vim by Tim Pope
• vim-license-gen (aka vim-license.dev)
Slide 95
Slide 95 text
No content
Slide 96
Slide 96 text
Merged !!
Slide 97
Slide 97 text
No content
Slide 98
Slide 98 text
No content
Slide 99
Slide 99 text
Wait for Licensee Update
Slide 100
Slide 100 text
No content
Slide 101
Slide 101 text
Wait for Licensee Release
Slide 102
Slide 102 text
No content
Slide 103
Slide 103 text
Wait for GitHub to Upgrade
Slide 104
Slide 104 text
No content
Slide 105
Slide 105 text
Wait and Wait
• Give zero pressure to the GitHub
• Wait and wait
• Then one day, I test the Licensee again
Slide 106
Slide 106 text
No content
Slide 107
Slide 107 text
No content
Slide 108
Slide 108 text
The Cause
• The change of the similarity algorithm
• No test case for Vim License
Slide 109
Slide 109 text
Fix Licensee
Slide 110
Slide 110 text
The Solution
• Update the algorithm or manual decision parameter
• Modi
fi
ed a parameter from 10 to 13
• Add test case for Vim License
Slide 111
Slide 111 text
No content
Slide 112
Slide 112 text
The Result
• Mike Linksvayer already create a PR to
fi
x the issue
• For each substitution
fi
eld, provide 2 ch space not counting to the penalty
• Merged at Oct 2020
Slide 113
Slide 113 text
No content
Slide 114
Slide 114 text
Wait Again
Slide 115
Slide 115 text
No content
Slide 116
Slide 116 text
Feb 2021
Slide 117
Slide 117 text
GitHub Support
Slide 118
Slide 118 text
No content
Slide 119
Slide 119 text
No content
Slide 120
Slide 120 text
Wait Again and Again
Slide 121
Slide 121 text
No content
Slide 122
Slide 122 text
Jan 2022
Slide 123
Slide 123 text
GitHub Search by License
Slide 124
Slide 124 text
No content
Slide 125
Slide 125 text
Vim’s License Still Not Correct
Slide 126
Slide 126 text
No content
Slide 127
Slide 127 text
No content
Slide 128
Slide 128 text
GitHub Support
Slide 129
Slide 129 text
No content
Slide 130
Slide 130 text
No content
Slide 131
Slide 131 text
No content
Slide 132
Slide 132 text
No content
Slide 133
Slide 133 text
No content
Slide 134
Slide 134 text
Summary
Slide 135
Slide 135 text
• 2 issues
• 5 pull requests
• 1 mailing-list thread
• 1 new web app
• 2 GitHub support request
• ~ 5 years
Slide 136
Slide 136 text
Take away
Slide 137
Slide 137 text
• How to start a CONTRIBUTION and how to communicate
• What is SPDX and SPDX License List
• What is Licensee and how it work
• The deep understand of Vim License
Slide 138
Slide 138 text
Thanks for You Listening
Slide 139
Slide 139 text
Questions?