SPDX Wiki - User contributions [en]

Legal Team/Minutes/2018-01-11

2018-01-12T03:07:24Z

Brad.edmondson: /* Agenda */ wording

== Attendees ==
* Steve Winslow
* Gary O’Neall
* Jilayne Lovejoy
* Brad Edmondson
* Dennis Clark
* Alan Tse

== Agenda ==
1) 3.0 release - plan to do 3.1 around end of Jan, what should we aim to tidy up?
* LLVM exception - should hear from them soon (JL has been in contact)
* assembly exception that Wayne requested - need to follow-up (JL to follow-up)
* AGPL-1.0-only and AGPL-or-later need to be added
* automated testing and deployment: any time anyone commits anything into repo (whether merged, PR, etc.) a test runs to see if it is valid (e.g.,. missing end tags, or misspelled XML tag) & will run test to run a match against license it is based on. Gary has seen a lot of the latter, this way person submitting can fix it. Typical errors to this end he’s seen are just structural (XML tags) or typos; most of matching errors are due to trying to get too fancy with alt/var tags; some are problems with tools themselves.
** only change will be that when you add a new license, you need to add text of license in
* there is also an automatic deployment PR - so that when a change is merged, it will automatically updates the license-list-data repo and then when there is a release tagged. (this will be really helpful for Gary!)
** question as to automatic tagging or review before tagging - do we want some kind of formal review of license-list-data before we way that is complete (there isn’t now)
** there is also question as to updating website - should that be automated as well? could push to preview site for every commit and then for live site for a tagged release automatically. All agreed that preview push is kind of nice, as we can see things in advance and fix weird formatting issues, but maybe hold off on automatic release for tagged releases for time being. Also consider using release candidates in between and use that for review.

other thoughts (that are not urgent, need to be dealt with for 3.1):
* should we test for broken URLs as part of test run? this is not part of test now, could add this later
** we might also add a note in the field list on Overview page that we can’t control external URLs - ok to PR to do so; but need to make sure new link is actually same license and/or use web archive for original link (may not want to dictate one over the other?)

Process for reviewing issues and PRs:
* plan to change who can merge directly. we had more people on the XML repo at first, to get through initial review. (not that we don't trust people, but have too many right now and some of those people aren't actively contributing at this point in time.) JL to sort out, may need advice from more Github-savvy folks!
* identify process of who’s doing what, etc. - discussed how tech team operates for its repos:
** each repo has main owner - in our case, we really have one and probably need designated reviewers/committers for both technical issues and legal issues, preferably at least 2 of each kind so we don't have logjams (or 3)
*** Jilayne, Brad for legal; Gary, and maybe Alexios? (Gary to reach out) - these people would have decision making say, or can boot to calls if more discussion is wanted/needed/desired
** some PRs may be obvious, minor, don't need discussion - committers can mark for next release milestone and just merge at their discretion (always add milestone, so that is consistent)
** use calls to go through new PRs on call to decide what release and assign tasks; and then actual work done off call (for PRs)
*** Dennis offered to be assigned to anything content oriented
* use labels for things that need to be discussed, are mostly legal or mostly technical, etc.
* need to write this process up after a bit more feedback - and post where?

went through some of issues open and set milestones, assigned with rest of time on call

Legal Team/Templatizing/ActionPlan

2017-08-04T20:21:42Z

Brad.edmondson: /* General Cleanup / Future Tag Changes */

==Action Items - Stage 1a: XML Conversion of Existing Licenses==

=== Existing License PRs ===
* Finish main review of all licenses so that all can merged (Who: ALL)
* Kris to put original "conversion" tool (to convert text files into XML format, to then be reviewed) in public repository so anyone else can use it or improve it (KRIS) -- DONE
** can be found here: https://github.com/myndzi/license-tool see further notes on this below

=== Exceptions ===
* Convert and add exceptions to repository (KRIS) - DONE
* note from Gary: tools don't care whether in same or different directory --> they are in their own directory
* Review exceptions and merge exception PRs (ALL) - DONE

=== Deprecated licenses ===
* outcome: Proposed solution to add a boolean attribute named deprecated to the SPDX element with a default value of false. For the "+" deprecated licenses, we would copy the XML file, rename it to include the "+" and add deprecated=true to the SPDX attributed.
* Convert and add deprecated licenses to repository (GARY) - almost done
* Review and merge deprecated license PRs (ALL) 

=== PRs tagged "requires acknowledgement" or "has self-referring numbering"===
* Originally, we planned to add XML tags to reflect these characteristics (and not merge these licenses until we do so). For now, though, we will leave this as a potential feature for the future, and proceed to merge these licenses as long as their basic formatting looks correct. (We will be able to go back and review which PRs had these labels even after merging.)
* Merge those labeled "approved" and "requires acknowledgement" (JILAYNE) - DONE
* In future, may also need a more thorough review of existing licenses to see if any others meet these criteria (i.e. can't rely solely on the labeled PRs)

=== True-up license list to current release ===
* Need to true-up changes to existing licenses from 2.3 (when XML repo was created) to current 2.6 release using spreadsheet (JILAYNE)
* Convert and add new licenses to repo from 2.4, 2.5. 2.6 (KRIS or someone from tech team using Kris' tool) - DONE
* Review and merge PRs for new licenses/exceptions since 2.3 (ALL) - DONE

=== General Cleanup / Future Tag Changes ===
* After review is complete and all licenses and exceptions have been merged, general clean-up to include: (GARY) -->
* Initial round of clean-up to include:
** remove body tag
** fix lower case spdx tags
** get rid of HTML-escaped characters where not needed / keep those that are needed (e.g., greater than / less than symbols)
** update XML tag names as per tech team recommendations (see below)
** pretty-print XML
* Clean-up or formatting issues to be considered later:
** <s>does all text need to be wrapped in tags? or could titles just be <title>license title</title> instead of <title>license title</title>?</s> Yes, [https://github.com/spdx/license-list-XML/issues/414#issuecomment-320094253 everything needs] and tags
** add back the tags Brad removed / generally prettify licenses so that they display as close to original as possible
** check license families (CC-BY, GPL, etc.) for consistent markup (e.g., what is optional in CC-BY licenses?)
** add a tag for "deprecated licenses", possibly add more definitive advice on alternative license expression

===Tracking license list version for new licenses===
* when a new license is added to the list is currently recorded in the new license request list and in the SPDX License List master spreadsheet. Where will this info be in the new XML world? should we put this meta data in license itself, as a new tag or can we use Github release magic to track that way? --> discuss at OSLS, see minutes here: http://wiki.spdx.org/view/Legal_Team/Minutes/2017-02-15-OSLS
** added new attribute for this, but will only implement for 2.6 and onward, need to check on attribute name - In Google doc
* Add attribute to new licenses as of v2.6 - DONE

=== Finalize XML tag names ===
* Gary to bring to tech team for Sept 20 call, tech team to decide, with final review/sanity check by legal team - DONE
** process: build a table for existing field name (machine-readable name for these from current spec) and new XML tag - DONE
** Gary took to tech team, and had joint call with legal and tech team to finalize. Notes here: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-10-25 and https://docs.google.com/document/d/1z9n44xLH2MxT576KS_AbTOBtecyl5cw6RsrrQHibQtg/edit
* update tags as per tech team recommendations (action item noted above) as part of "General Clean-up" in above section

== Action Items - Stage 1b: Switch to Github, using old license list format - Jan 30 ==
* Update info for README on Github for initial switch to Github from Git repo, prior to XML conversion (Dennis to draft, ALL review) - DONE
** completed - see here: https://docs.google.com/document/d/1kGSs3CXbtjF_uZmVsjhXMGChyP-pzlszktTr8t4PmmI/edit#
* update README on Jan 30 or after with this text 
* update webpages where needed: (JILAYNE w/help from rest of legal team) - DONE
** links to master files
** page on requesting a new license - how to update, just point to README?
** what else?
* what to do with existing license request tracking spreadsheet: will need to move any "live" items into issue tracking in Github; leave where it is otherwise as dormant? download and post somewhere on wiki to be safe?
** where does data as to what version a license gets added to list go going forward? (see above)

== Action Items - Stage 2: Go live with XML conversion==

=== Check website generation and tools ===
* As of Sept 16 call, Gary had done a first pass on tools to generate website and caught a bunch of validation errors, many of which were from lower case <spdx> tag. Gary to see what other validations were and fix and merge if simply or create pull request/submit to legal team if needs review
* other updates to spdx-tools for new XML format to generate the website along with the 5 different license formats in available in the license-list-data git repository (Gary) 
* preview of website before release
* make sure everything is in there (i.e., diff on text files that they are identical)

=== Documentation ===
* Review various webpages for updating
** Alan did first pass, which raised various questions and other things to address, need to revisit now that naming issue has been resolved with tech team
** license matching guidelines web page
** license list overview web page
** README in repo and info in repo itself
** request a new license web page
** Appendix in spec:
*** Appendix 1 is list of licenses - check
*** Appendix 2 must be updated
*** Appendix IV and V - check for links, references, etc.
** other pages?
** info about XML tags (http://wiki.spdx.org/view/Legal_Team/Templatizing/tags-matching) - this may become part of spec as an appendix, but where to put in meantime?

=== Coordination with tech team and SPDX Spec v2.2 ===
* revisions to current appendix in spec as to markup (e.g., var v. alt markup tag)
* explanation of XML tags - to go in markup appendix or new appendix?
* v2.2 of spec is not going out this year, so may want to think about stuff that we might want to go into a new appendix eventually, where should it go in the meantime and plan for a migration?
* ---> discuss at OSLS

=== Incorporating Synonyms into XML license documents ===
* Kris had a plan to do this, but not sure he can contribute that now.
* will postpone this until someone has a plan and can do the work, but will include in the .xsd schema
* 2 options: all within license text itself or via a file that's stored in repository and lists them all in a structured way tools can use
* will need someone to work on this

=== Create new tool to take text of new license and convert to XML file ===
* to do so by hand is a bit tedious. Will still need human review before finalizing. Initial tool Kris used probably won't be applicable. Would be nice if this could be web-based
* discuss with tech team, find someone to take this on? ---> discuss at OSLS

=== Tool for verifying XML against known matching/known non-matching text files ===
* Testing regime for XML (either manually or by continuous integration) to validate that XML source generates other formats correctly

Legal Team/Templatizing

2017-07-20T17:49:43Z

Brad.edmondson: /* Key Links */ add XML validator link

This is the working area for the technical and legal teams. The focus of this effort is for recording ideas and considerations about improving machine detection and recognition of licenses and headers in code.

== Key Links ==
* Bugzilla bug 1302, which started this particular conversation: https://bugs.linuxfoundation.org/show_bug.cgi?id=1302
* Meeting minutes from Aug 6, 2015 Legal call: http://wiki.spdx.org/view/Legal_Team/Minutes/2015-08-06
* Matching Guidelines: http://spdx.org/spdx-license-list/matching-guidelines
* Pull Request Proposal for SPDX License List (Gary): https://docs.google.com/document/d/115Lis1SJV7Rp-XuNjIysU61urzFGjUOBPqEdVyGLsfI/edit
* Meeting minutes from Oct 29, 2015 Legal call: http://wiki.spdx.org/view/Legal_Team/Minutes/2015-10-29
* Initial proposal, full write-up (Kris): http://wiki.spdx.org/view/Legal_Team/Template_proposal
* Meeting minute from Feb 9 joint tech/legal call: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-02-09
* Meeting minutes from Feb 18 legal call: http://wiki.spdx.org/view/Legal_Team/Minutes/2016-02-18
* Proposal for exact tag names and edits to matching guidelines: http://wiki.spdx.org/view/Legal_Team/Templatizing/tags-matching
* '''Plan for review of XML license files:''' http://wiki.spdx.org/view/Legal_Team/Templatizing/ReviewXML
* Action plan for whole project: http://wiki.spdx.org/view/Legal_Team/Templatizing/ActionPlan
* XML tags naming discussion:
** link to Oct 26 joint tech/legal team meeting minutes: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-10-25
** The working document is available at https://docs.google.com/document/d/1z9n44xLH2MxT576KS_AbTOBtecyl5cw6RsrrQHibQtg
* Link to XML conversion tool Kris wrote for initial conversion: https://github.com/myndzi/license-tool
* Kris' tool for seeing XML conversion: http://krisreeves.com/xmledit/
* XML validator: https://www.w3schools.com/xml/xml_validator.asp

== Background ==
(from JL) The current Matching Guidelines and license markup was developed over the past several years. While the guidelines were relatively easier to come up with, how much markup to use and how to implement it was an on-going discussion of much debate. As relevant to the proposals here, the discussion around how much markup to use recognized that more markup would increase consistent results by reducing the need for tool makers to interpret the guidelines. However, due to the amount of work of potentially having markup on every license file, a more conservative approach was taken with the recognition that we could always add more markup over time (but taking it away would be harder).

The template language was a result of a collaboration between members of the legal and technical teams. The goal of the template was developed with the following goals in mind:
* Enable both humans and, to a lesser extent, tools to be able to determine if two license texts were equivalent.
* Preserve the original language of the license text.
* Leverage existing annotations and tools (e.g. the use of regular expressions to denote matches to variable text).

The current templating language is documented in Appendix II of the specification (http://spdx.org/sites/spdx/files/SPDX-2.0.pdf)

* Please review the current Matching Guidelines here: http://spdx.org/spdx-license-list/matching-guidelines
* Also see Bugzilla bug 1302, which started this particular conversation: https://bugs.linuxfoundation.org/show_bug.cgi?id=1302
* Meeting minutes from Aug 6, 2015 Legal call: http://wiki.spdx.org/view/Legal_Team/Minutes/2015-08-06

== Proposal ==
(from Gary) As a result of the initial bug 1302 discussion, Gary suggested to enable pull requests to be made against the SPDX License List, which would encourage contributions to markup by others. Such pull requests/contributions would be moderated by the Legal team (especially regarding determinations of markup and potential impact on legal interpretation of the license) with help from the Tech team as needed. Everyone on the Aug 6 legal call agreed this would be a great approach. Gary, Jilayne, Kate have begun to discuss how to implement this. A process also needs to be outlined for how pull requests will be handled, etc. A rough draft process proposal can be found in the google document https://docs.google.com/document/d/115Lis1SJV7Rp-XuNjIysU61urzFGjUOBPqEdVyGLsfI/edit?usp=sharing

(from Kris) Things that would improve working with the data programmatically:
* Get away from spreadsheets as the index for the licenses. Use JSON or XML (leaning towards XML to keep the toolset the same as markup parsing). A tool to compile the spreadsheet (for humans to work with) down to an XML file would probably work.
* Put all information about a license in the same place for each license (headers, addons/exceptions/whatever). Likely within markup within the license files themselves.
* Use XML or some other widespread format for the markup. This ensures there are tools available to work with the data in any language without having to write custom parsing tools or use tools tied to a specific platform because they're the only ones that exist.
* As, and if, the data moves towards a more markup-heavy format, it might be wise to simply include the full text verbatim in its own section rather than "recreate" it from marked up data.

An example of what a more normalized (to xml) data set might look like:

index.xml:
<licenses>
<license identifier="Apache-2.0" file="Apache-2.0.xml">Apache License 2.0</license>
...
</licenses>

LGPL-2.0.xml:
<license>
<original>...full original text content...</original>
<template>
<header>...Licensed under the Apache License, version 2.0, etc....</header>
<body>Some text blah blah <match regex="foo|bar"/> more text</body>
</template>
</license>

An alternative:
<template>
<header>....etc</header>
<body>...THE SOFTWARE IS PROVIDED "AS IS" AND <alt regex="THE AUTHOR">ISC</match> DISCLAIMS ALL WARRANTIES...</body>
</template>

The above carries the useful property that the text content of the body tag is the original source without any modification; processing can instead replace "alt" tag contents with their regex attribute to "convert" to a matchable format.

I don't actually like XML, but it seems to be a good fit for this purpose. It helps keep the intrusiveness of the markup into the license text focused and to a minimum, lets us wrap entire sections as optional, keep the canonical original text if desired, and specify alternate license formats such as the short-form license header and any desired metadata all in one place. And there are tools to read XML in every language.

I recommend that the template content be *without* the parts that the matching guidelines say to ignore, and preferably it should have normalized or marked up bullets, etc. Best of all would be, simply, normalized text content in a full regular expression that can be used to match directly, rather than a bunch of markup that must be turned into a regular expression -- but I think this would make maintenance much harder. Again, a compilation step could help. There's a lot of work to be done here that can be qualified as preprocessing, and there's no reason we can't make sure the preprocessing is "done right" and publish the results, instead of requiring programmers to reimplement their own version of the preprocessing with uncertain results.

A particular note about normalizing text: Bullet items are one of the problematic points for parsing because it can be troublesome to distinguish a bullet point like "ii." from the end of a sentence that got wrapped to the start of the next line. (I actually encountered this in my testing; a saving grace is that the bullets tend to be things that are hard to confuse for words, but it's really easy to write a regular expression that isn't that smart.)

For what it's worth, I'm quite happy to contribute when it comes to tooling; I already have code to do the most of this anyway. -Kris

== Detailed proposal ==

See: [[Legal_Team/Template_proposal]]

== Use and Feedback re: Current Markup ==
Feedback was solicited on the legal and tech mailing lists to this end:

We have a task to add markup to some of the standard headers and have also had input to add/edit markup on existing licenses. As a result of the latter, it has been raised that perhaps the markup could be improved. Before adding more markup (to standard headers, license text or both), it seemed prudent to start a discussion as to whether the existing markup is effective. Please ponder the following questions:
a) have you used the existing markup for matching purposes?
i) if no, why not?
ii) if yes, has it been helpful/effective? Could it be improved, and if so, how? (this will likely involve putting forward a proposal for review)

Please record your input in a section here, by name or tool or both:

=== Gary O'Neall ===
For the SourceAuditor commercial tools, the markup is used to validate that 2 licenses are equivalent per the matching guidelines. The open source SPDX tools uses the markup is used in a number of ways. The "compareSpdx" and "compareMultipleSpdx" commands use the markup to determine if the licenses are equivalent. There are library methods implemented to compare license text and report if the license text matches any of the SPDX LicenseList.

In all cases above, the markup is used to compare 2 existing known license text. It is NOT used to match license text against a library of possible license matches. In the commercial tool, a separate algorithm implements this functionality and the markup language turned out to be too inefficient for this purpose - at least for the performance requirements of our application.

Note: When we originally discussed the markup language, we debated whether to cover the use case of searching a library of possible license matches and the decision was taken not to support this.

In my opinion, the markup works fine for matching two license texts. If we wanted to support a searching use case, we would need to modify/extend the markup language to enable this to be efficient.

Performance would be greatly improved if we bounded the RE's. Something which is within the spec, but is not implemented in the templates. I would support implementing bounding all RE's in an upcoming release and changing the spec to encourage bounding. I think we could pick a pretty practical upper bound that would improve the performance.

Also note - this is more of an issue with the "matching against a library of licenses" issue than simply comparing two license texts since the former is more performance sensitive.
=== Sam Ellis ===
I will share a few points from my experience in templatization. I currently use a different templatization syntax that predates SPDX, but the principle of using regular expressions embedded within the license text is similar.

The major barrier to me adopting the SPDX templates is insufficient templatization within the existing licenses. The SPDX templates currently encode what I perceive to be the ‘official’ variations, i.e. organization name, person name, product name etc. However, real-world licenses contain may minor variations that may be inconsequential from a legal perspective, but nonetheless do not warrant separating out as separate licenses. Here is an example from the GPL-3.0 notice where it is common to see two variations in one of the sentences:

distributed in the hope that it will be useful
distributed in the hope that they will be useful

The example above is fairly uncontroversial, I would hope. However, there are plenty of other examples that border on having a legal impact. For example, in these two BSD-2-Clause variations it is necessary to consider whether the additional word constitutes an acceptable minor variation or warrants a different classification altogether:

Redistributions of source code must retain the above copyright notice, this list of…
Redistributions of source code must retain the above copyright notice unmodified, this list of…

It is the grey cases like these that make expanding the use of templating difficult. Inevitably it leads to having to make some judgements about the impact of a particular word or phrase on the legal interpretation, something that I am aware SPDX tries to avoid.

Whether it is worth templating all the cases like these primarily depends on the goals of the SPDX templates. If they are for human use to see what official variations are permitted, then they are not necessary. On the other hand, if they are to be used by automated license scanning tools, then covering these cases is essential in order to have a tool that works effectively on real-world code. So I think an important point is to gain clarity on the purpose of the templates.

In terms of the current application of the templates, I have a technical concern over the use of unbounded regular expressions, for example:

<<var;name=copyrightHolderAsIs;original=THE COPYRIGHT HOLDERS AND CONTRIBUTORS;match=.+>>

This is unbounded because it will match any number of characters for the copyrightHolderAsIs field. The practical consequence of this is that regular expression matching can explode in terms of time. I don’t have a concrete example to hand, but my own experience with using the same unbounded regular expressions on real-world licenses is that I have seen it take minutes just to process one regular expression on a single file, and this does not scale well when there are millions of files to process. Clearly, in terms of English language there is no maximum size on the length of a copyright statement. Using an unbounded regular expression is therefore correct in theory but difficult to use in practice. I have had to use size bounded regular expressions in order to have a scanning tool that will complete in a reasonable time. The problem in switching to bounded regular expressions is in deciding on what is an acceptable upper bound on the size, and this can really only be judged by experimentation against real-world licenses, and does then require on-going tweaking as new license variations are discovered.

Neither of these are problems with templatization per-se, and they are more to do with the extent and way in which they are currently applied.
=== Philippe Ombredanne ===
a) have you used the existing markup for matching purposes?

Yes and No: ScanCode uses an SPDX-inspired/derived markup, but instead of reusing the markup directly from the main license texts, markup is transformed in a simpler {{mustache-like}} syntax added to copies of these texts used only for detection purpose.

Because:
- adding more markup to a reference license text makes this eventually no longer usable as a reference text and harder to read by humans
- the many variations found in the wild make it hard to put all in a single template.
- the markup syntax implies eventually an implementation using regular expressions. ScanCode does not use regex, but inverted indexes and string alignments.

ii) if yes, has it been helpful/effective? Could it be improved, and if so, how? (this will likely involve putting forward a proposal for review)

I think a simple markup is a very effective way to detect licenses with minor text variations and still call this an exact match. It is also a very effective way to indicate variations for humans. I find it hard personally to mix the human readability and technical detection concerns in the same file without compromises.

As food for thought, here are some examples of markup as used in ScanCode:

https://github.com/nexB/scancode-toolkit/blob/b37be4de78152fbd3ed54761627c960010ce26a3/src/licensedcode/data/rules/apache-1.1_38.RULE#L17
https://github.com/nexB/scancode-toolkit/blob/b37be4de78152fbd3ed54761627c960010ce26a3/src/licensedcode/data/rules/bzip2-libbzip-1.0.5_1.RULE#L1

The syntax is using double curly braces to enclose variable parts. There is no regex involved. Optionally a number can be used after the opening braces to indicate the number of variable words, defaulting to 5 words.
For instance {{ Copyright (c) 2015 Myco }} would match up to 5 words
and {{ 10 Copyright (c) 2015 Myco inc.}} would match up to 10 words.

I hope this helps even though this is a slightly different take.

=== Bill Schnineller ===
a) have you used the existing markup for matching purposes?
No.
i) if no, why not?
In a nutshell, partly due to timing of our dev efforts ahead of SPDX templatization rollout, partly due to performance in context of our internal Use Case of scanning every single open source file (not just a handful of applications).

Details:
Black Duck has a corpus of license variants extracted from our Knowledge Base of around half a billion unique open source source/text and binary files.
Several years ago, prior to SPDX license templatization, we went through multiple iterations of grouping license text variants, as well as license name/nickname variants.
Our groupings were based on applying similarity algorithms, followed by human review. Our methodology has been in place for some time prior to the templates / regular expressions subsequently rolled out by SPDX.

Variations we've encountered, such as the street address of an organization, or typos / word substitutions that SPDX license templates might not have covered are some of the reasons we haven't yet gone through the exercise to see which license variants the SPDX templates might not 'match' to the license id's we've grouped them under.

We use our license scanner to scan (and rescan) every single open source file to populate our Knowledgebase of file-level license data. Our scanner uses multiple techniques to discover license references, but does not use regular expressions because of performance concerns.

Because Black Duck does not provide legal advice, consumers of our tools are able review the license text which our tools highlight in order to make a final determination. This fits well with the SPDX concept of separating discovered and concluded information.

Outlook:
SPDX license templates / regexes aren't as useful or efficient as other matching techniques we have, when automatically bulk scanning large codebases to determine ‘LicenseFoundInFile'.

But when a human is in the loop producing a final SPDX Document with Concluded License, SPDX license templates / regexes could be useful to focus legal review on deviations which may or may not be significant.

=== Nuno Brito ===
a) have you used the existing markup for matching purposes?
i) if no, why not?

Didn't use the markup. At the time haven't found documented the pattern keywords to find inside each license term. For example, one can find [xxxx]-[xxxx], [Owner Organization], <AUTHOR>. However, didn't seemed to be used consistently throughout the list of licenses. That was my opinion at that time, might certainly be proven erroneous.

If help is welcome, I can volunteer for normalizing the data for the complete list (top to bottom) and document the used keywords. However, for our own tooling usage as template I'd replace the text on some specific license terms so that we could later generate new licence docs where only the copyright holder changes.

Such example would be http://spdx.org/licenses/BSD-3-Clause-Attribution.html where: "Universidad de Palermo, Argentina" (http://www.palermo.edu/).'" becomes "<AUTHOR>".

=== Kris Reeves ===
I have been thinking about the proposed format in detail, and it has become clear to me that it will be undesirable to attempt to maintain a canonical single source of truth in the format we would want to expose to consumers. This is made clearest by the synonym matching rule: it is unfeasible to ask or expect a person editing a license file to find, identify, and mark up all candidates for synonym replacement. It is, however, easily possible to expose data that has been thus marked-up to consumers of the SPDX data. This implies that there must be a build step of some sort in the mix.

Given that there would be a build step, the case for altering the source markup is weakened somewhat; it's not strictly *necessary* to support (my) goal of providing data that can be consumed directly without any programmatic logic required to match a license, but still retains a number of benefits including: utility in editing/managing the license content itself, ease of integration into tooling required to produce the "built" product, ready extension, familiarity to newcomers/potential contributors, and even brevity.

Given the above, there are really two classes of markup to consider:

1) Markup used to clarify / identify information that is obvious to a human but difficult for a computer to interpret (for example: identifying bullet points)

2) Markup used to provide more robust matching that is easy for a computer to generate but difficult for a human to maintain (for example: identifying synonymous word spellings)

The most useful focus for markup in the first category is ease of use by a human editor: it should be unobtrusive and succinct, lending to ready interpretation by a reader/editor of the license text in a legal-meaningful way and imposing a minimum of technical knowledge for correct and confident interpretation.

The most useful focus for markup in the second category is ease of consumption by a program: it should be thorough and explicit, lending to immediately useful applications with little to no business logic required in the consuming code.

I'm working on the proposal discussed in the last legal call, but splitting it into two parts by these guidelines. This approach feels a bit messy to me, but I see no way around it other than to write custom tooling that can abstract the category-2 markup away from a human and let them work in terms of category-1 markup.

=== Compatibility with Current Markup ===
If we do change the format in a way which is incompatible with the existing markup, we may break existing applications. I originally did not think this would be much of an issue based on the responses above, but then I ran across the comment https://github.com/sindresorhus/spdx-license-list/issues/6#issuecomment-150606727 where the existing syntax is used to extract the license text. There may be other such applications that we are not aware of. -- Gary O'Neall

This is actually a point in favor of going ahead: if people are writing regular expressions to parse the current data, that is happening because the tools to do it properly aren't available to them. Markup change would obviously be a major version bump, so tools such as the above would still work and continue to function on the v2.0 data. New/updated tools, however, would become less fragile by virtue of using parsing tools that know how to deal with XML, and the markup itself can thus be extended to incorporate new needs without breaking existing fragile regular expressions. -- Kris

== Markup to add/consider ==
[JL] This is simply a list of various specific licenses that may need additional markup or things to consider related to the matching guidelines, to ensure we don't forget to add/edit later!
* Apache-2.0 - instructions as to how to apply license, markup as optional? (from Kris)
* MIT - "THE AUTHORS OR COPYRIGHT HOLDERS" - markup to be variable (from DMG's student paper)
* add "noninfringment" and "non-infringement" and "non infringement" as equivalent words in Matching Guideline #8 list of words (from DMG's student paper)
* add "contributors" and "co-contributors" and "co contributors" as equivalent words in Matching Guideline #8 list of words, needs some discussion (from DMG's student paper)
* ISC license needs markup (and link fixed)
[[Category:Technical]] [[Category:Legal]]

Legal Team/or-later-vs-unclear-disambiguation

2017-05-30T15:01:00Z

Brad.edmondson: initialization and summary of the issue and potential resolutions

=== Problem ===
==== D. Wheeler 5/25/2017 ====
Technically “GPL-2.0” in SPDX means “only this version”, but in practice many practitioners & tools are sloppy about this. Part of the problem is that tools can easily determine that “GPL version 2.0 is in this package” but in many cases they cannot easily determine automatically a distinction between “2.0 or greater” versus “2.0 and no other”. In addition, in many cases it doesn’t matter, so the increased effort would be a waste of time. What the tools really need to indicate is a way in SPDX to indicate “2.0 at least is here, and I don’t know if ‘or later’ is okay”. Since SPDX doesn’t have a mechanism to report this, “GPL-2.0” is sometimes being used to report of “I know 2.0 is here, and I don’t know if ‘or later’ is okay” - even though it’s technically not compliant with the SDPX spec.

It’d be helpful to have a simple way to indicate “I really mean this specific version” (my “!” suffix) vs. “this version at least is okay, and I’m not sure about later versions” (which is how “GPL-2.0” is currently interpreted; maybe another suffix like “?” or “*” would help to mark this case).

=== Selected responses/ideas/solution proposals ===
''' NOTE:''' Please feel free to add/clarify below

==== K. Stewart 5/25/2017 ====
We've started having some discussions with FSF about what they'd prefer, and their preference seems to be GPL-2.0-only, so we probably want to go that way rather than introducing the "!" idea.

==== D. Wheeler 5/25/2017 ====
K. Stewart: We've started having some discussions with FSF about what they'd prefer, and their preference seems to be GPL-2.0-only, so we probably want to go that way rather than introducing the "!" idea.
Okay. Although that's less flexible, that's much easier to transition (you don't have to change any parsing code), so I see the advantages of this.

If this is done:
1. It needs to cover all the licenses where this is likely. At *least* GPL and LGPL; I think MPL is probably in this case too.
2. The original license terms need to *stay* in SPDX, with modified clarifying text. Something like this:

GPL-2.0:
The GNU General Public License (GPL), version 2.0 is acceptable, and without any clear statement if later versions are acceptable. Where practical, try to use more specific license expressions such as "GPL-2.0+", "GPL-2.0-only", or "(GPL-2.0-only OR GPL-3.0-only)". Historically this indicator meant "GPL version 2.0 only", but in practice tools often can't determine if later ones are acceptable (or not) & used this term in such cases. This specification acknowledges this practice and provides more specific alternatives when that information is available.

==== D. Wheeler 5/26/2017 ====
We need at least *3* cases. Here they are, with potential names/expressions:
* GPL-2.0-only. I *know* that *only* the GPL version 2.0 is acceptable. I had originally proposed a "!" suffix.
* GPL-2.0+. I *know* that GPL version 2.0, or later, is acceptable.
* GPL-2.0. I *know* that at least GPL version 2.0 is acceptable (e.g., I found its license text). However, I'm not entirely certain whether or not later versions are acceptable, so I make *no* assertion either way. This appears to be what "GPL-2.0" has become, in some cases, in spite of the spec. Which is why we need a way to mark certainty vs. uncertainty. If you prefer, you could label this "GPL-2.0-at-least", or add a "?" suffix to mean "I don't know if later/other versions are acceptable".

The problem is that while tools can detect the presence of a license, it's often difficult for them to determine if an "or later" clause is valid in some cases. In many cases SPDX is capturing tool output, so we need for there to be a valid expression for tools to output. My understanding is that some tools that find GPL version 2.0 will currently report "GPL-2.0"... even if a later version is also acceptable... and as a result, "GPL-2.0" is not being interpreted as originally intended.

What's more, without a third case, it'll just happen again. Tools can't easily determine if "or later" applies, and in many cases you do *NOT* need more information than this. It can take a lot of effort ($) to determine if it's really "GPL-2.0-only" or "GPL-2.0+", and if the spec only supports those two options, then that's a problem.. because people are *not* going to spend effort unnecessarily.

If "GPL-2.0" is deprecated, then tools will start reporting "GPL-2.0-only" when they're not sure if later versions apply, because in many cases they can't easily determine it. Then we'll be back to the original problem, where "GPL-2.0-only" may mean "I found GPL 2.0 but maybe later versions will be okay". Ugh. Since many tools can only determine "at least this version", there needs to be a standard way to report it.

==== T. King 5/26/2017 ====
Digging at this “acceptable” idea a bit more, I'm guessing it's something like “adapters may share adapted works under”. But the SPDX isn't just about copyleft (e.g. it includes CC-BY-ND-*). I think it makes more sense to focus on licenses (just the text, e.g. GPL-2.0) and license grants. For example, here are some SPDX License Expressions translated into grants:

* GPL-2.0: You can redistribute it and/or modify it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation.

* GPL-2.0+: You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

* CC-BY-SA-4.0: This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

You can distribute an adaptation under a later version of the CC BY-SA because that's part of the CC-BY-SA-4.0 [1].

* CC-BY-SA-4.0+: This work is licensed under a Creative Commons Attribution 4.0 International License; either version 4.0 of the License, or (at your option) any later version.

The CC-BY-SA-4.0 tries to grant you that right anyway, but regardless of how you read the CC-BY-SA-4.0, I'm granting you that right directly.

> CC-BY-SA-3.0+ would be a synonym for CC-BY-SA-3.0 [6], but I don't
> see a problem with that. It would probably be useful to call that
> out in the wording that forbids the -only suffix for CC-BY-SA-3.0…

If the SPDX doesn't want to get into the business of determining when licenses grant + semantics, then we probably don't want an -only suffix and we certainly don't want a GPL-2.0-only short identifier.

But if you want to be in the business of warning users about the lack of built-in or-later wording in the GPL, CC-BY-ND-4.0, etc. and the presence of built-in or-later in the CC-BY-SA-4.0, etc., I don't see how you'd avoid making claims about whether the license had built-in or-later wording.

==== D. Seaward 5/26/2017 ====
Perhaps "GPL-2.0"could be deprecated and instead whoever is tagging must use one of:
* GPL-2.0-only
* GPL-2.0+
* GPL-2.0?
...where "only" means only, "+" means "or later" and "?" means unclear. Legacy data still tagged "GPL-2.0" would be treated as "GPL-2.0?" until updated. (This assumes the SPDX team want people tagging things as unclear!)

Legal Team

2017-05-30T14:42:20Z

Brad.edmondson: add working page for -only vs -or-later vs unclear expression syntax

This is the working area for the Legal Team. The SPDX Legal Team supports and provides recommendations to the SPDX working groups regarding licensing issues for the specification itself; maintains the [http://spdx.org/licenses/ SPDX License List]; and promotes the SPDX specification to the legal community at-large.

The Legal Team meets '''every other Thursday at 18:00 GMT (10:00AM PT, 11:00 MT, 12:00 CT, 1:00PM ET)'''.
Call this number: (United States): +1-857-216-2871
User PIN: 38633
International: visit the URL at http://uberconference.com/SPDXTeam

* [[Legal_Team/Priorities|Current Priorities and Work in Progress for the Legal Team]]
* [[Legal_Team/Minutes|Meeting Minutes of the Legal Team]]
* [[Legal_Team/Templatizing|Discussions on Templatizing Licenses and Headers to improve machine detection & recognition]]
* [[Legal_Team/non-English-licenses|Working page for policy on how to handle non-English licenses and matching]]
* [[Legal_Team/or-later-vs-unclear-disambiguation|Working page for policy on disambiguating -only vs. or-later vs. unclear expression syntax (esp. GPL)]]
* [[Legal_Team/Decisions|Decisions of the Legal Team]]
* [[Legal_Team/Old|Older Items for the Legal Team]]

[[Category:Legal]]

Legal Team/Templatizing/ActionPlan

2017-02-21T19:17:37Z

Brad.edmondson: /* General Cleanup */ add consistency check for license families (e.g. CC-BY)

==Action Items - Stage 1a: XML Conversion of Existing Licenses==

=== Existing License PRs ===
* Finish main review of all licenses so that all can merged (Who: ALL)
* Kris to put original "conversion" tool (to convert text files into XML format, to then be reviewed) in public repository so anyone else can use it or improve it (KRIS) -- DONE
** can be found here: https://github.com/myndzi/license-tool see further notes on this below

=== Exceptions ===
* Convert and add exceptions to repository (KRIS) - DONE
* note from Gary: tools don't care whether in same or different directory
* Review exceptions and merge exception PRs (ALL)

=== Deprecated licenses ===
* outcome: Proposed solution to add a boolean attribute named deprecated to the SPDX element with a default value of false. For the "+" deprecated licenses, we would copy the XML file, rename it to include the "+" and add deprecated=true to the SPDX attributed.
* Convert and add deprecated licenses to repository (GARY) - almost done
* Review and merge deprecated license PRs (ALL) 

=== PRs tagged "requires acknowledgement" or "has self-referring numbering"===
* Originally, we planned to add XML tags to reflect these characteristics (and not merge these licenses until we do so). For now, though, we will leave this as a potential feature for the future, and proceed to merge these licenses as long as their basic formatting looks correct. (We will be able to go back and review which PRs had these labels even after merging.)
* Merge those labeled "approved" and "requires acknowledgement" (JILAYNE) - DONE
* In future, may also need a more thorough review of existing licenses to see if any others meet these criteria (i.e. can't rely solely on the labeled PRs)

=== True-up license list to current release ===
* Need to true-up changes to existing licenses from 2.3 (when XML repo was created) to current 2.6 release using spreadsheet (JILAYNE)
* Convert and add new licenses to repo from 2.4, 2.5. 2.6 (KRIS or someone from tech team using Kris' tool) - DONE
* Review and merge PRs for new licenses/exceptions since 2.3 (ALL)

=== General Cleanup ===
* After review is complete and all licenses and exceptions have been merged, general clean-up to include: (GARY) -->
** remove body tag
** fix lower case spdx tags
** get rid of HTML- and UTF8-escaped characters
** update tags as per tech team recommendations (see below)
** does all text need to be wrapped in tags? or could titles just be <title>license title</title> instead of <title>license title</title> ?
** add back the tags Brad removed / generally prettify licenses so that they display as close to original as possible
** check license families (CC-BY, GPL, etc.) for consistent markup (e.g., what is optional in CC-BY licenses?)
** ... others?

===Tracking license list version for new licenses===
* when a new license is added to the list is currently recorded in the new license request list and in the SPDX License List master spreadsheet. Where will this info be in the new XML world? should we put this meta data in license itself, as a new tag or can we use Github release magic to track that way? --> discuss at OSLS, see minutes here: http://wiki.spdx.org/view/Legal_Team/Minutes/2017-02-15-OSLS
** added new attribute for this, but will only implement for 2.6 and onward, need to check on attribute name
* Add attribute to 2.6 new licenses (???) 

=== Finalize XML tag names ===
* Gary to bring to tech team for Sept 20 call, tech team to decide, with final review/sanity check by legal team - DONE
** process: build a table for existing field name (machine-readable name for these from current spec) and new XML tag - DONE
** Gary took to tech team, and had joint call with legal and tech team to finalize. Notes here: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-10-25 and https://docs.google.com/document/d/1z9n44xLH2MxT576KS_AbTOBtecyl5cw6RsrrQHibQtg/edit
* update tags as per tech team recommendations (action item noted above) as part of "General Clean-up" in above section

== Action Items - Stage 1b: Switch to Github, using old license list format - Jan 30 ==
* Update info for README on Github for initial switch to Github from Git repo, prior to XML conversion (Dennis to draft, ALL review) - DONE
** completed - see here: https://docs.google.com/document/d/1kGSs3CXbtjF_uZmVsjhXMGChyP-pzlszktTr8t4PmmI/edit#
* update README on Jan 30 or after with this text 
* update webpages where needed: (JILAYNE w/help from rest of legal team) 
** links to master files
** page on requesting a new license - how to update, just point to README?
** what else?
* what to do with existing license request tracking spreadsheet: will need to move any "live" items into issue tracking in Github; leave where it is otherwise as dormant? download and post somewhere on wiki to be safe?
** where does data as to what version a license gets added to list go going forward? (see above)

== Action Items - Stage 2: Go live with XML conversion==

=== Check website generation and tools ===
* As of Sept 16 call, Gary had done a first pass on tools to generate website and caught a bunch of validation errors, many of which were from lower case <spdx> tag. Gary to see what other validations were and fix and merge if simply or create pull request/submit to legal team if needs review
* other updates to spdx-tools for new XML format to generate the website along with the 5 different license formats in available in the license-list-data git repository (Gary) 
* preview of website before release
* make sure everything is in there (i.e., diff on text files that they are identical)

=== Documentation ===
* Review various webpages for updating
** Alan did first pass, which raised various questions and other things to address, need to revisit now that naming issue has been resolved with tech team
** license matching guidelines web page
** license list overview web page
** README in repo and info in repo itself
** request a new license web page
** Appendix in spec:
*** Appendix 1 is list of licenses - check
*** Appendix 2 must be updated
*** Appendix IV and V - check for links, references, etc.
** other pages?
** info about XML tags (http://wiki.spdx.org/view/Legal_Team/Templatizing/tags-matching) - this may become part of spec as an appendix, but where to put in meantime?

=== Coordination with tech team and SPDX Spec v2.2 ===
* revisions to current appendix in spec as to markup (e.g., var v. alt markup tag)
* explanation of XML tags - to go in markup appendix or new appendix?
* v2.2 of spec is not going out this year, so may want to think about stuff that we might want to go into a new appendix eventually, where should it go in the meantime and plan for a migration?
* ---> discuss at OSLS

=== Incorporating Synonyms into XML license documents ===
* Kris had a plan to do this, but not sure he can contribute that now.
* will postpone this until someone has a plan and can do the work, but will include in the .xsd schema
* 2 options: all within license text itself or via a file that's stored in repository and lists them all in a structured way tools can use
* will need someone to work on this

=== Create new tool to take text of new license and convert to XML file ===
* to do so by hand is a bit tedious. Will still need human review before finalizing. Initial tool Kris used probably won't be applicable. Would be nice if this could be web-based
* discuss with tech team, find someone to take this on? ---> discuss at OSLS

=== Tool for verifying XML against known matching/known non-matching text files ===
* Testing regime for XML (either manually or by continuous integration) to validate that XML source generates other formats correctly

Legal Team/Templatizing/ActionPlan

2017-02-16T23:53:55Z

Brad.edmondson: /* True-up license list to current release */ add text-cleanup tasks and make general clean-up separate from true-up

==Action Items - Stage 1a: XML Conversion of Existing Licenses==

=== Existing License PRs ===
* Finish main review of all licenses so that all can merged (Who: ALL)
* Kris to put original "conversion" tool (to convert text files into XML format, to then be reviewed) in public repository so anyone else can use it or improve it (KRIS) -- DONE
** can be found here: https://github.com/myndzi/license-tool see further notes on this below

=== Exceptions ===
* Convert and add exceptions to repository (KRIS) - DONE
* note from Gary: tools don't care whether in same or different directory
* Review exceptions and merge exception PRs (ALL)

=== Deprecated licenses ===
* outcome: Proposed solution to add a boolean attribute named deprecated to the SPDX element with a default value of false. For the "+" deprecated licenses, we would copy the XML file, rename it to include the "+" and add deprecated=true to the SPDX attributed.
* Convert and add deprecated licenses to repository (GARY) - almost done
* Review and merge deprecated license PRs (ALL) 

=== PRs tagged "requires acknowledgement" or "has self-referring numbering"===
* Originally, we planned to add XML tags to reflect these characteristics (and not merge these licenses until we do so). For now, though, we will leave this as a potential feature for the future, and proceed to merge these licenses as long as their basic formatting looks correct. (We will be able to go back and review which PRs had these labels even after merging.)
* Merge those labeled "approved" and "requires acknowledgement" (JILAYNE) - DONE
* In future, may also need a more thorough review of existing licenses to see if any others meet these criteria (i.e. can't rely solely on the labeled PRs)

=== True-up license list to current release ===
* Need to true-up changes to existing licenses from 2.3 (when XML repo was created) to current 2.6 release using spreadsheet (JILAYNE)
* Convert and add new licenses to repo from 2.4, 2.5. 2.6 (KRIS or someone from tech team using Kris' tool) - DONE
* Review and merge PRs for new licenses/exceptions since 2.3 (ALL)

=== General Cleanup ===
* After review is complete and all licenses and exceptions have been merged, general clean-up to include: (GARY) -->
** remove body tag
** fix lower case spdx tags
** get rid of HTML- and UTF8-escaped characters
** update tags as per tech team recommendations (see below)
** does all text need to be wrapped in tags? or could titles just be <title>license title</title> instead of <title>license title</title> ?
** add back the tags Brad removed / generally prettify licenses so that they display as close to original as possible
** ... others?

===Tracking license list version for new licenses===
* when a new license is added to the list is currently recorded in the new license request list and in the SPDX License List master spreadsheet. Where will this info be in the new XML world? should we put this meta data in license itself, as a new tag or can we use Github release magic to track that way? --> discuss at OSLS, see minutes here: http://wiki.spdx.org/view/Legal_Team/Minutes/2017-02-15-OSLS
** added new attribute for this, but will only implement for 2.6 and onward, need to check on attribute name
* Add attribute to 2.6 new licenses (???) 

=== Finalize XML tag names ===
* Gary to bring to tech team for Sept 20 call, tech team to decide, with final review/sanity check by legal team - DONE
** process: build a table for existing field name (machine-readable name for these from current spec) and new XML tag - DONE
** Gary took to tech team, and had joint call with legal and tech team to finalize. Notes here: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-10-25 and https://docs.google.com/document/d/1z9n44xLH2MxT576KS_AbTOBtecyl5cw6RsrrQHibQtg/edit
* update tags as per tech team recommendations (action item noted above) as part of "General Clean-up" in above section

== Action Items - Stage 1b: Switch to Github, using old license list format - Jan 30 ==
* Update info for README on Github for initial switch to Github from Git repo, prior to XML conversion (Dennis to draft, ALL review) - DONE
** completed - see here: https://docs.google.com/document/d/1kGSs3CXbtjF_uZmVsjhXMGChyP-pzlszktTr8t4PmmI/edit#
* update README on Jan 30 or after with this text 
* update webpages where needed: (JILAYNE w/help from rest of legal team) 
** links to master files
** page on requesting a new license - how to update, just point to README?
** what else?
* what to do with existing license request tracking spreadsheet: will need to move any "live" items into issue tracking in Github; leave where it is otherwise as dormant? download and post somewhere on wiki to be safe?
** where does data as to what version a license gets added to list go going forward? (see above)

== Action Items - Stage 2: Go live with XML conversion==

=== Check website generation and tools ===
* As of Sept 16 call, Gary had done a first pass on tools to generate website and caught a bunch of validation errors, many of which were from lower case <spdx> tag. Gary to see what other validations were and fix and merge if simply or create pull request/submit to legal team if needs review
* other updates to spdx-tools for new XML format to generate the website along with the 5 different license formats in available in the license-list-data git repository (Gary) 
* preview of website before release
* make sure everything is in there (i.e., diff on text files that they are identical)

=== Documentation ===
* Review various webpages for updating
** Alan did first pass, which raised various questions and other things to address, need to revisit now that naming issue has been resolved with tech team
** license matching guidelines web page
** license list overview web page
** README in repo and info in repo itself
** request a new license web page
** Appendix in spec:
*** Appendix 1 is list of licenses - check
*** Appendix 2 must be updated
*** Appendix IV and V - check for links, references, etc.
** other pages?
** info about XML tags (http://wiki.spdx.org/view/Legal_Team/Templatizing/tags-matching) - this may become part of spec as an appendix, but where to put in meantime?

=== Coordination with tech team and SPDX Spec v2.2 ===
* revisions to current appendix in spec as to markup (e.g., var v. alt markup tag)
* explanation of XML tags - to go in markup appendix or new appendix?
* v2.2 of spec is not going out this year, so may want to think about stuff that we might want to go into a new appendix eventually, where should it go in the meantime and plan for a migration?
* ---> discuss at OSLS

=== Incorporating Synonyms into XML license documents ===
* Kris had a plan to do this, but not sure he can contribute that now.
* will postpone this until someone has a plan and can do the work, but will include in the .xsd schema
* 2 options: all within license text itself or via a file that's stored in repository and lists them all in a structured way tools can use
* will need someone to work on this

=== Create new tool to take text of new license and convert to XML file ===
* to do so by hand is a bit tedious. Will still need human review before finalizing. Initial tool Kris used probably won't be applicable. Would be nice if this could be web-based
* discuss with tech team, find someone to take this on? ---> discuss at OSLS

=== Tool for verifying XML against known matching/known non-matching text files ===
* Testing regime for XML (either manually or by continuous integration) to validate that XML source generates other formats correctly

Legal Team/Templatizing/ActionPlan

2017-01-19T19:14:40Z

Brad.edmondson: /* Action Items - Stage 2: Go live with XML conversion */ testing tool recommendation

Legal Team/Templatizing/ActionPlan

2017-01-19T18:41:16Z

Brad.edmondson: /* Check website generation / tools */ --- add testing

==Action Items - Stage 1a: XML Conversion of Existing Licenses==

* '''Existing License PRs'''
** Finish main review of all licenses so that all can merged (Who: ALL)
* Kris to put original "conversion" tool (to convert text files into XML format, to then be reviewed) in public repository so anyone else can use it or improve it (KRIS)
* '''Exceptions'''
** Convert and add exceptions to repository (KRIS or someone from tech team using Kris' tool) 
** note from Gary: tools don't care whether in same or different directory
** Review exceptions and merge exception PRs (ALL)
* '''Deprecated licenses'''
** outcome: Proposed solution to add a boolean attribute named deprecated to the SPDX element with a default value of false. For the "+" deprecated licenses, we would copy the XML file, rename it to include the "+" and add deprecated=true to the SPDX attributed.
** Convert and add deprecated licenses to repository (GARY) 
** Review and merge deprecated license PRs (ALL) 
* '''PRs tagged "requires acknowledgement" or "has self-referring numbering"'''
** Originally, we planned to add XML tags to reflect these characteristics (and not merge these licenses until we do so). For now, though, we will leave this as a potential feature for the future, and proceed to merge these licenses as long as their basic formatting looks correct. (We will be able to go back and review which PRs had these labels even after merging.)
** Merge those labeled "approved" and "requires acknowledgement" (JILAYNE) --> DONE
** In future, may also need a more thorough review of existing licenses to see if any others meet these criteria (i.e. can't rely solely on the labeled PRs)
* '''True-up license list to current release'''
** Need to true-up changes to existing licenses from 2.3 (when XML repo was created) to current 2.6 release using spreadsheet (?? NAME)
** Convert and add new licenses to repo from 2.4, 2.5. 2.6 (KRIS or someone from tech team using Kris' tool) 
** Review and merge PRs for new licenses/exceptions since 2.3 (ALL)
* '''General Cleanup'''
** After review is complete and all licenses and exceptions have been merged, general clean-up to include: (WHO? someone from tech team? discuss at OSLS) -->
** remove body tag
** fix lower case spdx tags
** get rid of HTML- and UTF8-escaped characters
** ... others?
** update tags as per tech team recommendations (someone from tech team)

=== Finalize XML tag names ===
* Gary to bring to tech team for Sept 20 call, tech team to decide, with final review/sanity check by legal team - DONE
** process: build a table for existing field name (machine-readable name for these from current spec) and new XML tag - DONE
** Gary took to tech team, and had joint call with legal and tech team to finalize. Notes here: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-10-25 and https://docs.google.com/document/d/1z9n44xLH2MxT576KS_AbTOBtecyl5cw6RsrrQHibQtg/edit
* update tags as per tech team recommendations (see above)

=== Documentation ===
* Update info for README on Github for initial switch to Github from Git repo, prior to XML conversion (Dennis to draft, ALL review) - DONE
** completed - see here: https://docs.google.com/document/d/1kGSs3CXbtjF_uZmVsjhXMGChyP-pzlszktTr8t4PmmI/edit#
* Review various webpages for updating (changes to be made now? or what until stage 2?) - Alan did first pass, which raised various questions and other things to address, need to revisit now that naming issue has been resolved with tech team
** license matching guidelines page
** license list overview page,
** info about XML tags (http://wiki.spdx.org/view/Legal_Team/Templatizing/tags-matching) - this may be more likely stage 2 and become part of spec as an appendix

== Action Items - Stage 1b: Switch to Github, using old license list format - Jan 29 ==
* update README (see above for link to text)
* update any relevant webpages:
** links to master files
** page on requesting a new license - how to update, just point to README?
** what else?

== Action Items - Stage 2: Go live with XML conversion==

=== Create new tool to take text of new license and convert to XML file ===
* to do so by hand is a bit tedious. Will still need human review before finalizing. Initial tool Kris used probably won't be applicable. Would be nice if this could be web-based
* discuss with tech team, find someone to take this on? ---> discuss at OSLS

=== Documentation: Define a new process for License List via Github and with new XML format ===
* i.e., how to request a new license, what kind of pull requests we want, underlying process etc.
* included checking with Kate re: moving license list to Github primary (instead of LF Git and mirrored to Github)

=== Check website generation / tools ===
* As of Sept 16 call, Gary had done a first pass on tools to generate website and caught a bunch of validation errors, many of which were from lower case <spdx> tag. Gary to see what other validations were and fix and merge if simply or create pull request/submit to legal team if needs review
* other updates to spdx-tools for new XML format to generate the website along with the 5 different license formats in available in the license-list-data git repository (Gary) 
* preview of website before release
* Testing regime for XML (either manually or by continuous integration) to validate that XML source generates correct text

=== Coordination with tech team and SPDX Spec v2.2 ===
* revisions to current appendix in spec as to markup (e.g., var v. alt markup tag)
* explanation of XML tags - to go in markup appendix or new appendix?
* ---> discuss at OSLS

Legal Team/Templatizing/ActionPlan

2017-01-18T19:29:01Z

Brad.edmondson: update action plan w/ my understanding from yesterday's huddle

==Action Items - Stage 1a: XML Conversion of Existing Licenses==

* '''Existing License PRs'''
** Finish main review of all licenses so that all can merged (Who: ALL)
* Kris to put original "conversion" tool (to convert text files into XML format, to then be reviewed) in public repository so anyone else can use it or improve it (KRIS)
* '''Exceptions'''
** Convert and add exceptions to repository (KRIS or someone from tech team using Kris' tool) 
** note from Gary: tools don't care whether in same or different directory
** Review exceptions and merge exception PRs (ALL)
* '''Deprecated licenses'''
** outcome: Proposed solution to add a boolean attribute named deprecated to the SPDX element with a default value of false. For the "+" deprecated licenses, we would copy the XML file, rename it to include the "+" and add deprecated=true to the SPDX attributed.
** Convert and add deprecated licenses to repository (GARY) 
** Review and merge deprecated license PRs (ALL) 
* '''PRs tagged "requires acknowledgement" or "has self-referring numbering"'''
** Originally, we planned to add XML tags to reflect these characteristics (and not merge these licenses until we do so). For now, though, we will leave this as a potential feature for the future, and proceed to merge these licenses as long as their basic formatting looks correct. (We will be able to go back and review which PRs had these labels even after merging.)
** Merge those labeled "approved" and "requires acknowledgement" (?? NAME) 
** In future, may also need a more thorough review of existing licenses to see if any others meet these criteria (i.e. can't rely solely on the labeled PRs)
* '''True-up license list to current release'''
** Need to true-up changes to existing licenses from 2.3 (when XML repo was created) to current 2.6 release using spreadsheet (?? NAME)
** Convert and add new licenses to repo from 2.4, 2.5. 2.6 (KRIS or someone from tech team using Kris' tool) 
** Review and merge PRs for new licenses/exceptions since 2.3 (ALL)
* '''General Cleanup'''
** After review is complete and all licenses and exceptions have been merged
** remove body tag
** fix lower case spdx tags
** get rid of HTML- and UTF8-escaped characters
** ... others?
** update tags as per tech team recommendations (someone from tech team) --> discuss at OSLS

=== Finalize XML tag names ===
* Gary to bring to tech team for Sept 20 call, tech team to decide, with final review/sanity check by legal team - DONE
** process: build a table for existing field name (machine-readable name for these from current spec) and new XML tag - DONE
** Gary took to tech team, and had joint call with legal and tech team to finalize. Notes here: http://wiki.spdx.org/view/Technical_Team/Minutes/2016-10-25 and https://docs.google.com/document/d/1z9n44xLH2MxT576KS_AbTOBtecyl5cw6RsrrQHibQtg/edit
* update tags as per tech team recommendations (see above)

=== Check website generation / tools ===
* As of Sept 16 call, Gary had done a first pass on tools to generate website and caught a bunch of validation errors, many of which were from lower case <spdx> tag. Gary to see what other validations were and fix and merge if simply or create pull request/submit to legal team if needs review
* other updates to spdx-tools for new XML format to generate the website along with the 5 different license formats in available in the license-list-data git repository (Gary) 
* preview of website before release

=== Documentation ===
* Update info for README on Github for initial switch to Github from Git repo, prior to XML conversion (Dennis to draft, ALL review) - DONE
** completed - see here: https://docs.google.com/document/d/1kGSs3CXbtjF_uZmVsjhXMGChyP-pzlszktTr8t4PmmI/edit#
* Review various webpages for updating (changes to be made now? or what until stage 2?) - Alan did first pass, which raised various questions and other things to address, need to revisit now that naming issue has been resolved with tech team
** license matching guidelines page
** license list overview page,
** info about XML tags (http://wiki.spdx.org/view/Legal_Team/Templatizing/tags-matching) - this may be more likely stage 2 and become part of spec as an appendix

== Action Items - Stage 1b: Switch to Github, using old license list format - Jan 29 ==
* update README (see above for link to text)
* update any relevant webpages:
** links to master files
** page on requesting a new license - how to update, just point to README?
** what else?

== Action Items - Stage 2: Go live with XML conversion==

=== Create new tool to take text of new license and convert to XML file ===
* to do so by hand is a bit tedious. Will still need human review before finalizing. Initial tool Kris used probably won't be applicable. Would be nice if this could be web-based
* discuss with tech team, find someone to take this on? ---> discuss at OSLS

=== Define a new process for License List via Github and with new XML format ===
* i.e., how to request a new license, what kind of pull requests we want, underlying process etc.
* included checking with Kate re: moving license list to Github primary (instead of LF Git and mirrored to Github)

=== Coordination with tech team and SPDX Spec v2.2 ===
* revisions to current appendix in spec as to markup (e.g., var v. alt markup tag)
* explanation of XML tags - to go in markup appendix or new appendix?
* ---> discuss at OSLS

=== Documentation ===
* see stage 1 list - any further updates?

=== SPDX tools and website generation ===
* see stage 1 list - any further updates?