SPDX Wiki - User contributions [en]

GSOC/GSOC ProjectIdeas

2019-01-10T17:15:07Z

Bkuhn: /* SPDX Specification in MarkDown */ I can mentor this one if the other mentors have other projects they're focused on.

'''Welcome to the 2019 SPDX Google Summer of Code Project Page'''

See the [https://rtgdk.github.io/spdx-gsoc-proposal.html proposal template] if you are interested in submitting a Google Summer of Code proposal.

Should you have questions please do not hesitate to contact one of the mentors directly.

 

__TOC__

 

== What is SPDX ? ==

First and foremost we are a community dedicated to solving the issues and problems around Open Source licensing and compliance. The SPDX work group (part of the Linux Foundation) consists of individuals, community members, and representatives from companies, foundations and organizations who use or are considering using the SPDX standard. The work group operates much like a meritocratic, consensus-based community project; that is, anyone with an interest in the project can join the community, contribute to the specification, and participate in the decision-making process. We come from many different backgrounds including open source developers, lawyers, consultants and business professionals, many of who have been involved with license compliance and identification for years.

As part of this effort we have developed a set of collateral that can be used:

* [https://spdx.org/using-spdx License List and Short Identifiers]
* [https://spdx.org/using-spdx SPDX Specification for generating SPDX Doucments in either RDF or Tag/Value format]
* [https://spdx.org/tools A set of basic tools for working with SPDX Documents]
* [https://spdx.org/using-spdx License Identifiers in source]

== Why choose an SPDX Project? ==

Contributing to one of the SPDX projects below will provide a valuable contribution to developers and/or users of open source software. We believe you will find the projects both technically challenging and rewarding. In essence we believe you will be able to look back one day and I say I was part of that effort.

 

= Getting Involved =

Beyond working wth your mentor(s) we highly encourage students who select one of these projects to get involved with the SPDX community via our technical working group. Interaction with the technical team is primarily done via its mailing list (see resources). There is however a weekly call you could join as well. All of the daily work for the Tech team is done on this wiki.

== Resources ==

* [http://spdx.org SPDX website]
* [https://spdx.org/specifications SPDX Specification]
* [https://spdx.org/tools SPDX Workgroup Tools webpage]
* [https://lists.spdx.org/mailman/listinfo/spdx-tech SPDX tech mailing list]

=SPDX Workgroup Tooling Projects=
These projects are aimed at contributing to the SPDX tools to help reduce the effort to create SPDX and increase the accuracy of the SPDX documents.

==Update Parser Libraries to SPDX 2.1 for GO==
Update one of the SPDX GO libraries to the SPDX 2.1 specification. The SPDX 2.1 specification is a major upgrade from SPDX 1.2 supporting relationships between SPDX documents and SPDX elements.
====Skills Needed====
* Development skills in the GO language
* Experience with parser development
* Understanding of RDF and XML
====Background Information====
SPDX currently provides libraries supporting the reading and writing of SPDX document. Currently, only Java libraries support the new SPDX 2.1 specification. The Python libraries and the GO libraries support version 1.2 of the spec. The libraries must support both RDF/XML import/export as well as tag/value import/export. The [[https://github.com/spdx/tools-go SPDX git repository]] SPDX Tools project contains the source code for the libraries.

====Available Mentors====
[mailto:gary@sourceauditor.com Gary O'Neall]

==Additional Format Support for the Python Libraries==
Add the ability to read and write XML, JSON, and YAML formats of the SPDX documents.

====Skills Needed====
* Development skills in the Python language
* Experience with parser development
* Understanding of XML, JSON and YAML

====Background Information====
SPDX 2.1 specification supports reading and writing RDF/XML and a tag/value format for SPDX documents. Version 2.2 of the specification will add support for XML, JSON and YAML. The Python libraries currently support reading and writing the RDF/XML and tag/value. This project would extend the parsing and file generation capabilities of the python libraries to include XML, JSON and YAML format.

The current python libraries are in the [[https://github.com/spdx/tools-python SPDX python tools git repository]]

====Available Mentors====
[mailto:tetechris20@gmail.com Krys Nuvadga], [mailto:gary@sourceauditor.com Gary O'Neall]

== Port SPDX license expression library to Ruby, JavaScript and Java==
The [[https://github.com/nexB/license-expression/]|licens_expressionlibrary]] provides comprehensive support license expression using a boolean engine for Python.
The goal of this project is to port and/or package this library for JavaScript, Ruby and Java, considering either code conversion tools, alternative Python implementations (e.g. Jython) or calling Python from another language to bring the same features to these other languages.
====Skills Needed====
* Development skills in Python, Java, Ruby, JavaScript.

====Background Information====
See https://github.com/spdx/tools-python/issues/10 and https://github.com/nexB/license-expression/
====Available Mentors====
[mailto:pombredanne@nexb.com Philippe Ombredanne]

=SPDX Specification Projects=
The following projects contribute directly to the creation or validation of the SPDX 2.1 specification.

== SPDX Specification in MarkDown ==
Migrate the specification from Google docs to GitHub+MarkDown based toolchain capable of generating HTML, PDF and EPUB
====Skills Needed====
* Understanding of documentation tooling
* Web-development skills to style HTML version
====Background Information====
The [https://spdx.org/specifications 2.1 SPDX specification] PDF and HTML version have several issues.
1. Navigation through both document is difficult as a index is missing
2. Switching to GitHub+MarkDown will remove friction for contributors to comment/amend the specification. Common workflow within the OSS community
====Available Mentors====
[mailto:kstewart@linuxfoundation.org Kate Stewart]
[mailto:thomas.steenbergen@here.com Thomas Steenbergen]
[mailto:bkuhn@sfconservancy.org Bradley M. Kuhn]

== SPDX Specification Wiki Examples of Package Managers ==
SPDX specification describes on a high level how to describe package, files and snippets but lack examples how to capture the use of package managers
====Skills Needed====
* Understanding of package managers
====Background Information====
To encourage adoption of SPDX it should be clear how to encode the use of common programming language package managers within SPDX. The aim of this project is to create example per build tool/package manager so that not only as example to the community but also form the input for SPDX tech team discussions and future tooling development

Initial package managers:
* Bower
* CocoaPods
* Gradle
* gem
* gitmodules
* Maven
* npm
* PyPi
* sbt
* NuGet

====Available Mentors====
[mailto:thomas.steenbergen@here.com Thomas Steenbergen]
[mailto:stewart@linux.com Kate Stewart]

== SPDX Specification Views for legal counsels and developers ==
The proposal is to see if it possible to deduct large SPDX documents into a small subset SPDX document providing a specific reduced "views" on larger data.
====Skills Needed====
* Understanding of compliance needs of legal counsels and developers so we can remove friction to adopt SPDX
====Background Information====
SPDX documents commonly contain 100s, if not 1000s of entries making it hard for a human to make manual corrections or draw conclusions. No scanner can provide 100% complete data human corrections are usual needed. The aim from this proposal is twofold:
1. Enable developers with a "code view" of tool-generated SPDX document close to the code they work on to enable them to make corrections to the SPDX data. For instance amend SPDX package tag values or model package dependencies not detected by used scanner.
2. Provide legal counsels with a "package and limited file view" to enable legal conclusions
====Available Mentors====
[mailto:thomas.steenbergen@here.com Thomas Steenbergen]
[mailto:ybronshteyn@blackducksoftware.com Yev Bronshteyn]

Legal Team/only-operator-proposal

2017-09-07T22:39:44Z

Bkuhn: /* Proposed Solution: add only operator */ FSF was not on most of the calls; Conservancy was not represented: I appeared in my own capacity and made that clear

==Introduction to Issue ==
===Historical Background ===
Originally, the SPDX License List listed variations of licenses as separate "line items". Notably, various versions of L/GPL had two listed items: one for a specific version-only, and one indicating a version-or-later. This was indicated in the list via the inclusion of the words "only" and "or later" in the full name; and via the short identifier: <code>GPL-2.0</code> and <code>GPL-2.0+</code> respectively. (examples throughout this page use GPL-2.0, but this issue applies to the entire family of GNU licenses, i.e., GPL, LGPL, FDL, APGL).

For example, the pre-version-2.0 SPDX License List looked like this:

GNU General Public License v2.0 only GPL-2.0
GNU General Public License v2.0 or later GPL-2.0+

When the license expression syntax was introduced in version 2.0 of SPDX, licenses with the "or later" indication were removed as separate listed licenses ("deprecated"), as the "or later" option could now be provided for via using the <code>+</code> operator. This enabled the use of the <code>+</code> operator with other licenses, as applicable.

The ability to create license expressions using the <code>+</code> operator along with the <code>with</code> operator also solved the issue of under-representation of license-exception combinations.

However, this also created a new issue whereby the standard header which is usually where the "or later" or "only" option is indicated in practice could no longer be differentiated or captured directly. This also resulted in the full license name having "only" with no real way to modify that to "or later" in the full license name where/when the <code>+</code>.

The original argument for the <code>+</code> operator was that it could be used with other licenses (not just GNU family). However, to anyone's memory we did, not at that time, conduct a full analysis of the other licenses with "or later" language or how they work in practice.

===Other License with "or later" Clauses===

Not all licenses that provide for later versions treat their application in the same way or as explicitly as the GNU family of licenses does.
A list of the licenses with text related to later versions, relevant text, and an analysis of the licenses that have "or later version" type language are listed on this page: https://wiki.spdx.org/view/Legal_Team/later-version-clauses

Notably, most licenses that reference the possibility of later versions can be read to say you can take and redistribute the work under the license you found it with or any other later version with no explicit option of 'this version only.'

The CDDL family is slightly different in that the CDDL is “or later” by default and “only” with [https://github.com/spdx/license-list-XML/blob/7ecb7363bc82aedd0e293ca8825e348181619e6a/src/CDDL-1.1.xml#L279-L289 an explicit notice prohibiting later versions]. There's currently no <code>only</code> operator for the CDDL, although some code (e.g. [https://github.com/freebsd/freebsd/blob/2c31a4b74c2e41b0c7407c9830e22bfd07150af0/uts/common/fs/zfs/abd.c#L2-L5 parts of FreeBSD's ZFS implementation]) does declare that prohibition while other code (e.g. [https://github.com/illumos/illumos-gate/blob/a9bfd41d542f15c474711abb8b0ca66a4cef9918/usr/src/common/acl/acl_common.h#L2-L19 some illumos userspace tools]) does not (and is therefore presumably CDDL-1.0+).

===Issue===
In practice, practitioners are not always explicit about whether a license is a version only or or later option. Also, because the "only" or "or later" distinction often is found outside the actual license text (e.g., in a license notice or header in the source files), machines (and humans) need a way to identify the license file itself without determining (yet) if the package is "only" or "or later". In other words, there is currently no way to express 'I just found this license text and am not sure what the package license is' using only license expressions.

The SPDX specification provides a way to distinguish between what is [https://spdx.org/spdx-specification-21-web-version#h.111kx3o detected license files] and the [https://spdx.org/spdx-specification-21-web-version#h.2lwamvv concluded file license] as well as similar fields at the package level when creating an SPDX document. But before getting to that point, tools must have ways to correctly identify what is precisely found in the files without having to draw a conclusion. Also, as discussed in the 2017-08-08 meeting, some ecosystems restrict themselves to license expressions (e.g. [https://docs.npmjs.com/files/package.json#license npm allows SPDX 2.0 license expressions]), and some tools attempt to detect package licenses by looking only at the license files and not at license-grant blurbs (e.g. [https://github.com/benbalter/licensee/blob/v9.2.0/docs/what-we-look-at.md Licensee], which is [https://developer.github.com/v3/licenses/ used by the GitHub license API]).

===Examples / Challenges===
The scenarios below map out various examples and how one would identify the [https://spdx.org/spdx-specification-21-web-version#h.111kx3o License Information in File] using SPDX short identifiers and bearing in mind machine reading for this task. Some of these scenarios have been discussed on the various calls on this topic as to how one would identify the license short identifier for each file currently (and under the proposal, see below). In any case, these scenarios need to have clear indication as to appropriate interpretation:

# '''you find: 1 text file with the license text of GPL-2.0; and 4 source files with the standard header for GPL-2.0, which include the language "any later version" '''
## 1 text file with license text of GPL-2.0 = <code>GPL-2.0</code>
## 4 source files with standard header for GPL-2.0, which include the language "any later version" = <code>GPL-2.0+</code>
## Concluded package license = <code>GPL-2.0+</code>
## Note: in the current reality, <code>GPL-2.0</code> for the license text file would mean "only", yet one cannot determine from that file alone if the copyright holder intends to use the version only or the or later option. Also note, that a machine can positively identify the "or later" option via the use of recommended standard license header.
# '''you find: 1 file with the license text of GPL-2.0; and 4 source files with the standard header for GPL-2.0, which omits the language "any later version" '''
## 1 text file with license text of GPL-2.0 = <code>GPL-2.0</code>
## 4 source files with standard header for GPL-2.0, which omits the language "any later version" = <code>GPL-2.0</code>
## Concluded package license = <code>GPL-2.0</code>
## Note: Same comment as #1 re: license text file. Also note, that a machine can positively identify the "only" option via the use of recommended standard license header which omits the "any later version" text.
# '''you find: no license file; 4 source files with the statement, “this is licensed under GPL" '''
## 4 source files = <code>GPL-1.0+</code>
## Concluded package = <code>GPL-1.0+</code>
## Note: The determination of <code>GPL-1.0+</code> would most likely need to be made by a human. But given the language of the GPL stating, "If the Program does not specify a version number of the license, you may choose any version ever published by the Free Software Foundation." and no version indicated in any manner whatsoever, we can feel reasonably confidant that any version can be applied and the short identifier <code>GPL-1.0+</code> covers any version.
# '''you find: 1 license file with GPL-2.0 license text; 4 source files with no license information whatsoever'''
## 1 text file with license text of GPL-2.0 = <code>GPL-2.0</code>
## 4 source files with no license information whatsoever = NONE (as per SPDX specification)
## Concluded license = ??
## Question: Is including a copy of a particular version of the license "the Program specif[ying] a version number of the license which applies to it" and thus we can confidantly conclude the package is GPL-2.0+?
## Answer: John Sullivan was [[Legal_Team/or-later-vs-unclear-disambiguation#Alternative_Solution_-_Do_Not_Deprecated_GPL-2.0|going to check for a FSF position on this]]. I'm not clear on what his reported findings were, but I expect they decided not to take a formal position on it.

==Goals==
* Provide way to get to "GPL-2.0-only" (or the like) identifier to enable better clarity for that option/scenario
* Ensure we accommodate the reality of other licenses with language re: later versions such that all options can be represented appropriately
* Ensure that there is some amount of consistency with what the short identifiers mean / Don't completely break the meaning for current users

==Proposed Solution: add <code>only</code> operator==
Based on the various legal and tech calls and including representatives from the LF (some notes in [[Legal Team/or-later-vs-unclear-disambiguation]]), we coalesced around the following proposal:

'''Create an "only" operator defined as to be used to indicate 'only the version of the license is intended for use' '''

For licenses that may include “only” or “or later” (or both) clauses in the license text, having explicit <code>+</code> and <code>only</code> operators to use in SPDX identifiers reduces the chance that an unfamiliar user mis-uses or misinterprets the identifier and provides a way to be explicit and accurate that was arguably not available before. This solution allows the license text to speak for itself (via the "plain" license identifier) and uses the <code>only</code> and <code>+</code> operators to be explicit in intent.

This leaves the ability to use the "plain" license identifier (without an operator) to indicate the existence of the license text itself or other scenarios where it is not clear whether the "or later" or "only" options are articulated by the copyright holder. This avoids SPDX having to make an interpretation as to what the licenses mean or intend. This would also bring into consistency the use of "plain" identifiers for all other licenses that have "or later" clauses.

The + operator would remain with the same definition.

Necessary/associated changes:
* add new "only" operator to Appendix IV: SPDX License Expressions of spec and explanatory language as to how/when to use it.
* Remove "only" from full name of GNU family licenses on SPDX License List
* clarify that the license ID used by itself would indicate that the license text itself should be used to determine if later versions of the license could be used (in Appendix IV or V?)

Potential issues:
* Backwards compatibility for GPL-2.0 meaning 'only' to now meaning 'whatever the license says' creates some inconsistency as the default position of the license text with no other info is "or later" or "any version", not "only". However, it was pointed out on the call that those people conscientious enough to be using GPL-2.0 correctly, will be more likely to be conscientious enough to add the <code>only</code> operator as needed once it's available. All seemed to agree that use of GPL-2.0 is probably not thought through as to "or later" or "only", and thus in these situations, there is no different meaning.
* Does not make clear indication of "ambiguous" situation to push people towards using 'only' and '+' operators to be explicit (preferred). However, we don't have a clear indication of ambiguity now either (is ambiguity ever clear?). You can clearly detect the ambiguous situations in tooling if you add metadata to license definitions recording whether they are compatible with the various version-related operators (as discussed [[Legal Team/or-later-vs-unclear-disambiguation#Alternative_Solution_-_create_-only_and_PROXY_.7BTEXT.7D_operators_for_licenses_that_explicitly_declare_their_compatibility|here]] and [https://lists.spdx.org/pipermail/spdx-legal/2017-August/002126.html here]).
* GPL language is clear that it is one or the other, so this could create more confusion and not reflect the intention of the license. However, all agreed that when you have a machine scanning code with no other license info other than a file with the license text of GPL-2.0, this is not entirely clear situation.