Revision as of 16:29, 4 April 2019

Welcome to the 2019 SPDX Google Summer of Code Project Page

See the proposal template if you are interested in submitting a Google Summer of Code proposal.

Should you have questions please do not hesitate to contact one of the mentors directly.

What is SPDX ?

First and foremost we are a community dedicated to solving the issues and problems around Open Source licensing and compliance. The SPDX work group (part of the Linux Foundation) consists of individuals, community members, and representatives from companies, foundations and organizations who use or are considering using the SPDX standard. The work group operates much like a meritocratic, consensus-based community project; that is, anyone with an interest in the project can join the community, contribute to the specification, and participate in the decision-making process. We come from many different backgrounds including open source developers, lawyers, consultants and business professionals, many of who have been involved with license compliance and identification for years.

As part of this effort we have developed a set of collateral that can be used:

• License List and Short Identifiers
• SPDX Specification for generating SPDX Doucments in either RDF or Tag/Value format
• A set of basic tools for working with SPDX Documents
• License Identifiers in source

Why choose an SPDX Project?

Contributing to one of the SPDX projects below will provide a valuable contribution to developers and/or users of open source software. We believe you will find the projects both technically challenging and rewarding. In essence we believe you will be able to look back one day and I say I was part of that effort.

Getting Involved

Beyond working wth your mentor(s) we highly encourage students who select one of these projects to get involved with the SPDX community via our technical working group. Interaction with the technical team is primarily done via its mailing list (see resources). There is however a weekly call you could join as well. All of the daily work for the Tech team is done on this wiki.

Resources

• SPDX website
• SPDX Specification
• SPDX Workgroup Tools webpage
• SPDX tech mailing list

SPDX Workgroup Tooling Projects

These projects are aimed at contributing to the SPDX tools to help reduce the effort to create SPDX and increase the accuracy of the SPDX documents.

Registry and Repository of License List Namespaces

Build automation for a GitHub repository to support registration of license namespaces to support the following workflow:

• User submits a request for a new license namespace
  • The namespace can be a dns-style request or a free-format namespace (e.g. .spdx.org.ad-hoc-licenses or this-is-my-licenses) as described in the SPDX spec namespace issue
• User includes optional information for a URL which resolves to an XML SPDX document which either contains all license references within the document or is a list of external SPDX references for the license definitions for the namespace
• User includes optional information on the submitter name and email
• User includes optional notes
• User agrees that the information will be publicly shared per the terms of the Linux Foundation privacy policy
• A check is made that the namespace is not already in use
• A pull request is created for the new namespace
• A committer to the namespace repository accepts the pull request
• When accepted, the namespace is published to a known website
• REST based API's are available to query the namespace repository

The namespace registration UI can be built as a function of the SPDX online tools or as an independent tool or as extensions to GitHub.

To support organizations who implement namespaces, provide a library which can implement a user interface for submitting and managing licenses for the namespace. The library can assume a github repository will be used to store the SPDX XML files which describe the licenses.

The library would support a web based UI and a REST API. It would support the following features:

• Add a license text to a known git repository. Input would be license text, license name, proposed license ID, optional license namespace, optional comment, optional creator. This would be stored as an SPDX XML document which defines the license. There would be one document per license.
• When added, the license would be checked to see if it already exists within the repository.
• Wen added, the license text would be compared to the SPDX listed licenses and report any possible duplicates.
• A single SPDX document will be updated/maintained that contains external references to the individual SPDX Files for the individual licenses - acting like a table of contents

Skills Needed

• Development skills in the Python or JavaScript language
• Understanding of GitHub API's
• Ability to work with the user community in refining requirements
• UI development
• REST API development

Background Information

SPDX provides a license list for commonly used open source license - the SPDX License List. SPDX also supports defining licenses within the SPDX document using a LicenseRef syntax defined in section 6 of the SPDX specification. In the next release of SPDX, we plan to introduce a mechanism for other organizations or individuals to maintain lists of licenses outside of the SPDX license list, but allow those licenses to be valid without requiring the text to be in the SPDX document itself. This enhancement has been documented in the SPDX specification issues list. This project automates the registration and management of the namespaces.

Available Mentors

Gary O'Neall, Mark Atwood

Develop a Distributed License Repository Management Application

Develop an application which accepts links to repositories of SPDX licenses and maintains information on the collection of all licenses references in the repositories. The interface the application would be a REST API. The application could also include a web based user interface. The application would periodically monitor the external repositories for any updates to the licenses. The application would support the following use cases:

• See if a license text has already been registered or if license text is already on the SPDX License List.
• See if the license text for a license matches license text for other licenses within the same repository.
• See if the license text for a license matches license text for other licenses within other repositories.
• Maintain a list of license aliases, preferably as a file in a github repositories. The aliases would include all license ID's for licenses with the same text.
• Provide a service that allows for text to be compared against all existing licenses.
• Promote a license to the license list - this would call the REST API's for the online tool to add a license to the SPDX license list.
• Remove a license repository. This would also update the license aliases.
• Provide metrics on use for licenses to help the SPDX legal team propose licenses which should be on the SPDX license list.

Background Information

See the above project idea "Registry and Repository of License List Namespaces" for background on the license name spaces. This project provides additional support for managing the namespace.

Skills Needed

• Development skills in the Python, Java or JavaScript language
• Understanding of Github API's
• Ability to work with the user community in refining requirements
• REST API development

Available Mentors

Gary O'Neall

Enhanced Workflow for Online License Request

Update the SPDX Online Tools license submit feature to support the following workflow:

• License submit can be initiated directly from the UI or through an external application (e.g. the SPDX License Diff browser plugin) using a documented API
• License text is compared to the currently approved license list
  • If matched, the SPDX ID is returned and the user is informed that the license already exists
• License is compared to all submitted yet not approved licenses
  • If matched, the user is informed the license is already submitted and is provided a link to the License List XML issue
• License is compared to all submitted and rejected licenses
  • If a match is found, the user is provided a link to the License List XML issue
• License is compared to the existing license list using an algorithm which finds close matches (SPDX License Diff is an example)
  • If an existing license is close, a diff view will show the word differences
  • The user is presented with a choice of adding an issue for the nearly matching license stating that the license should match
    • If the user chooses to add the issue, the license text will be added to the issue requesting a change to the license XML to allow the match
    • We could also implement suggested XML markup (e.g. alt or optional text) to make the licenses match - NOTE: This may be a technically challenging feature to implement
• If the user wants to submit a new license request, the information is captured and processed by the SPDX legal team through GitHub

Skills Needed

• Development skills in the Python language
• Experience with parser development
• Experience proposing spec changes
• Understanding of Github API's
• Experience in XML parsing

Background Information

The SPDX legal team uses an online request process for new license requests. This feature was implemented by a GSoC student in 2018. Extending the functionality to check for duplicate requests and checking for near matches would greatly improve the efficiency of the license request and approval process.

This project would require significant interaction with the users of the tool (the SPDX legal team) and would have some interesting technical challenges in storing and matching text. The optional feature of suggesting XML markup for near matches could involve sophisticated matching techniques to find the appropriate text to include as optional or alternate.

The current SPDX license list request process is documented in the License List XML contributing page.

Available Mentors

Gary O'Neall

Update Parser Libraries for Golang

A new Golang library has recently been added to the SPDX tools, at [1]. This tool updated the SPDX Golang libraries to the SPDX 2.1 specification. This tool has several opportunities for improvement and adding features.

Skills Needed

• Development skills in the Golang language
• Experience with parser development
• Understanding of RDF and XML

Background Information

SPDX currently provides libraries supporting the reading and writing of SPDX documents. A recent new tool has been added for parsing, generating and working with SPDX documents in Golang [2]. Opportunities for improving and adding features to this tool include the following:

• adding support for the official RDF format
• experimenting with support for other formats, such as JSON, YAML and XML
• enabling support for parsing and generation of documents under pre-2.1 versions of the SPDX spec

Available Mentors

Steve Winslow Gary O'Neall

Additional Format Support for the Python Libraries

Add the ability to read and write XML, JSON, and YAML formats of the SPDX documents.

Skills Needed

• Development skills in the Python language
• Experience with parser development
• Understanding of XML, JSON and YAML

Background Information

SPDX 2.1 specification supports reading and writing RDF/XML and a tag/value format for SPDX documents. Version 2.2 of the specification will add support for XML, JSON and YAML. The Python libraries currently support reading and writing the RDF/XML and tag/value. This project would extend the parsing and file generation capabilities of the python libraries to include XML, JSON and YAML format.

The current python libraries are in the [SPDX python tools git repository]

Available Mentors

Krys Nuvadga, Gary O'Neall

Port SPDX license expression library to Ruby, JavaScript and Java

The [[3]|licens_expressionlibrary]] provides comprehensive support license expression using a boolean engine for Python. The goal of this project is to port and/or package this library for JavaScript, Ruby and Java, considering either code conversion tools, alternative Python implementations (e.g. Jython) or calling Python from another language to bring the same features to these other languages.

Skills Needed

• Development skills in Python, Java, Ruby, JavaScript.

Background Information

See https://github.com/spdx/tools-python/issues/10 and https://github.com/nexB/license-expression/

Available Mentors

Philippe Ombredanne

SPDX Specification Projects

The following projects contribute directly to the creation or validation of the SPDX 2.1 specification.

SPDX Specification in PDF and HTML

We need to generate both HTML and PDF versions of the SPDX Specification from Markdown (MD). The default is English but going forward we envision other language translations as well so they would need to be accounted for in the overall structure and approach.

What we need going forward is:

• The ability to generate both an HTML and PDF document for every released version of the specification. This would be a final copy that should should not be re-rendered.
• The ability to display "real time" draft versions of the specification in HTML. This means changes to the specification via commits to GIT for that version should be shown, More than one draft version should be able to be displayed in this fashion. That means we could be rendering draft version 3.1 and draft version 4.0. We do not need this in PDF.
• For the PDF version we will need a cover page, page numbers, TOC, header and footer. Internal document references should work (see the background). The ideal solution will be to come up with something that works for both the HTML and PDF versions un-changed but some automated post processing for the PDF is okay and changing how th elinks are done in the MD is viable as well.
• The default is English but going forward we envision other language translations as well for the HTML and PDF so they would need to be accounted for in the overall structure and approach.
• The specifications are currently released on branches today. The structure of the GIT may need some changes to accommodate language versions and the PDF.

Other approaches than what we are using in the background can be suggested and used if they meet the requirements above, particularly for internal document links.

Skills Needed

• Understanding of documentation tooling: Markdown, HTML, mkdocs, etc.,.
• Familiarity with GIT and github API's

Background Information

The 2.1 SPDX specification has been moved to markdown on at https://github.com/spdx/spdx-spec and now generates an HTML version at: https://spdx.github.io/spdx-spec

If you look at the SPDX GitHub today it shows the 2.1 SPDX specification nicely as HTML. This is done using a Travis VM, GitHub Pages and markdown. Mkdocs is used to generate the HTML pages. You can look at the scripts in the GIT. What we need going forward is a way to generate both a PDF version and HTML and for each draft and release version of the specification.

For the 2.1.1 specification we are currently using pandoc and wkhtmltopdf to generate a PDF specification. This is currently being done offline. In doing this we need to combine the MD documents as each chapter in the specification is a separate MD document. That would work ecept there are internal links that go between the MD documents; e.g. xxxxxx/chapter4.md. These links are broken on conversion to the PDF so a solution for both HTML and PDF needs to be found.

A potential approach other than pandoc and wkhtmltopdf to consider for the PDF is at https://github.com/tombensve/MarkdownDoc.

Available Mentors

Jack Manbeck Kate Stewart Thomas Steenbergen

SPDX Specification Views for legal counsels and developers

The proposal is to see if it possible to deduct large SPDX documents into a small subset SPDX document providing a specific reduced "views" on larger data.

Skills Needed

• Understanding of compliance needs of legal counsels and developers so we can remove friction to adopt SPDX

Background Information

SPDX documents commonly contain 100s, if not 1000s of entries making it hard for a human to make manual corrections or draw conclusions. No scanner can provide 100% complete data human corrections are usual needed. The aim from this proposal is twofold: 1. Enable developers with a "code view" of tool-generated SPDX document close to the code they work on to enable them to make corrections to the SPDX data. For instance amend SPDX package tag values or model package dependencies not detected by used scanner. 2. Provide legal counsels with a "package and limited file view" to enable legal conclusions

Available Mentors

Steve Winslow Thomas Steenbergen

SPDX Document Generator for projects using SPDXIDs

As more projects start to use SPDXIDs at the file level it becomes much simpler to generate SPDX docs for them from a python script.

Skills Needed

• Ability to program in python

Background Information

Forward thinking open source projects are adopting SPDXIDs in source files (initially U-Boot, but now much wider use like Zephyr, Linux Kernel, etc.) With these easy to find "SPDX-License-Identifier:" strings, generating an SPDX document for a project is a matter of iterating over the files in a project and extracting the information from these SPDXIDs and calculating checksums. Creating an open source tool to do this will aid these projects in generating accurate SBOM information at release time. This tool should be implemented as a command line, so it can be incorporated into builds, and options can be added. Goal is that projects that use SPDX identifiers can automatically generate a SPDX document as a Software Bill of Materials (SBOM) on demand (build, release, etc.).

Available Mentors

Kate Stewart Uday Shankar