Skip to content

Latest commit

 

History

History
502 lines (344 loc) · 37 KB

README.md

File metadata and controls

502 lines (344 loc) · 37 KB
Raw

How to Open Source at Zalando

A guide to releasing an open-source project at Zalando, Europe's largest online fashion platform. Please feel free to use this as a template for your own organization's open source planning, policymaking, and development efforts. If there's a topic we've missed, or if you have a suggestion for making this better, let us know via our Issues tracker.

We're really grateful to Software Lead Weekly, cron.weekly, Chris Aniszczyk (TODO Group), the folks at Datio Engineering, Thomas Lockney of Nike, Richard Seroter at Pivotal, Peter Zaitsev at Percona, Jonathan Lipps at Sauce Labs, and others for spreading the word about this doc.

Table of Contents

###Why Open Source?

Because it can: improve quality, mitigate risk, increase trust, save us money, expand our technology choices, be fun, enable us to give back to the community, strengthen our tech brand, and attract talent.

###Our Open Source First Principles

Vision: We strongly believe that open source software benefits the tech community, and that providing broadly useful code to the world is a virtue. We strive to work in an open source way to the betterment of Zalando and the world.

  • Do “Open Source First”: If your Zalando project can also be useful to non-Zalandos, release it as open source from the start.
  • Take Ownership: Your team is responsible for ensuring that it’s possible to open-source your project. Your delivery lead is available for guidance.
  • Share Your Code: All code shared between teams must be open source.
  • Be Safe: To ensure the broadest possible use of your project, use the MIT License only.
  • Deliver Quality: Provide a great out-of-the-box experience.
  • Provide Documentation: Include a clear README and default working configuration.
  • Stay Secure: Make sure your project doesn’t include Zalando specifics, such as credentials and private identifiers.
  • Ask for Help: Find colleagues to brainstorm ideas for your project and to review your work.
  • Promote: Tell the world about your project via blog posts, social media and conference talks.
  • Join the Open Source Guild: Help us make open source stronger at Zalando! Come on, it'll be fun. :)

This blog post might also be of interest.

###Project Design

####What to Ask (and Answer) Before You Open Source Danese Cooper and Duane O'Brien at PayPal have shared a succinct four-question approach that has influenced our own:

  • Who cares?
  • Are we still using it?
  • Are we committed to it?
  • Can it be developed in one public tree?

To encourage our engineers to dive deeper on the first point, we've created this product template/checklist. It's meant to raise the key questions necessary for a project to reflect a strong "product mindset" and maximum project utility for the community at large.

####Never open-source these:

  • PCI DSS-related projects: e.g. payment services
  • Domain knowledge
  • Customer data
  • Unique Selling Points (USPs)
  • Anything that would risk our competitive advantage. This typically means technologies we build that are intrinsic to generating or reinforcing the uniqueness of our customer experience, and that—if made public—would enable our competitors to implement it and erase our uniqueness. This could be:
    • confidential source code
    • recommendation algorithms
    • search functionalities that give us an edge over competitors

If you're open-sourcing a project that has contained sensitive information in the past, the sensitive information can still be retrieved from the Git commit history. Create an entirely new Git repo for it before pushing it to GitHub.

No issues? Great! On to the next section ...

###Where to Publish: Main Org, Incubator, or GHE?

Based on quality, usefulness and maintenance considerations, we use this matrix to decide how to classify and place our projects:

  • InnerSource = GitHub Enterprise
  • Coding in the Open = Incubator
  • Open source = main Zalando org

In the inverse, you could think of these three categories as “steps” for your project as it develops from an idea to a minimum viable product (MVP) that someone on the outside could understand, use and possibly work on as a contributor.

The next sections offer more details on differences between open source, “coding in the open” and InnerSource.

####What Makes a Project “Open Source”

A Zalando project doesn't become "open source" simply by appearing on public GitHub. For a project to become open source as we define the term, it:

  • is useful beyond Zalando. It is free of Zalando dependencies and simple for a non-Zalando to install and start using.

  • has high-quality documentation that is up-to-date and clear about what the project does. (This template can help you.)

  • is tested. It has automated tests and takes advantage of test coverage.

  • is under active development, or is stable enough to be considered a “finished” product. If the project is incomplete, at least one maintainer has worked on it in the last three months. If it’s stable and doesn’t require constant maintenance, you’ve stated as much in your README.

  • is innovative. If it duplicates an existing project, it does at least one thing better, faster, differently, etc., or is higher-quality.

  • meets our non-negotiable guidelines regarding security and compliance.

  • is an MVP. It either meets or surpasses “minimum viable product” status. An outside developer could use it and even contribute to it. If it’s buggy or very early-stage, it includes a brief development status in the intro stating as much. (This template can help you.)

  • has a plan. Its maintainers care about making it a success. They commit to responding to PRs and issues in a timely manner (48-72 hours), thank contributors, and convert quality contributors to trusted maintainers as appropriate. If you need some guidance, check out Mozilla's helpful resources on this topic.

  • Example projects: Patroni, Connexion, Zappr, Python NSEnter, Tailor, Grafter, SwiftMonkey.

  • Where it belongs: Zalando main org

We have also published some document-only repositories like our RESTful API guidelines, our “How to Open Source” guidelines, the Tech Radar, and our Rules of Play. These reflect our organizational perspective on technical topics, and are therefore of potential interest to other companies who are looking for ideas and case studies. We should keep them limited in number and broad in scope.

What Makes a Project “Coding in the Open”

“Coding in the open” is how we describe a Zalando project that is already public—or that you want to make public for personal reasons—but not currently useful beyond Zalando Tech because it is affected by at least one of the following issues:

  • it is dependent on other Zalando projects

  • it is difficult or impossible for a non-Zalando to install

  • it has documentation that lacks essential user information, or lacks docs altogether

  • it isn’t currently maintained

  • it is in pre-MVP (minimum viable product) status

  • duplicates the functionality of an existing project and is of questionable quality (due to lack of code review or testing)

  • Example projects: oakkeeper, STUPS, stups2go, solution-center-login, Tarbela, Tranzlate

  • Where it belongs: Zalando Incubator

#####Quick Incubator FAQ

######Why did we make the Incubator? To support our Open Source First principle, "Share Your Code,” while reserving our main GitHub org for projects useful beyond Zalando.

######Can you launch a project directly to the Incubator? Yes. And you should, if it meets “coding in the open” criteria.

######Do Incubator projects have to follow our security and compliance guidelines? Yes, even if we are developing them for Zalandos only, “experimental,” etc.

######Can an Incubator project ever appear in the main Zalando GitHub org? Absolutely. Even if Incubator projects aren't currently useful outside of Zalando, we'd like to think that one day they might be.

######What happens when you transfer a repo from one org to another? The URL is re-routed to the new organization (zalando-incubator). All git clone, git fetch, or git push operations targeting the previous location will continue to function as if made on the new location. Links will be transferred to the new repository.

That said, GitHub strongly recommends that anyone who had a reference to the old link locally change to the new URL. Read GitHub’s documentation to learn more.

######What happens if I “code in the open” on the main org? The Guild will ask you to transfer the project yourself to the Incubator, first by filing an Issue on the project and then via HipChat/one-on-one. If we don’t get any response within 14 days, or if you agree to the transfer but don’t take action within a week (seven days), your project will be relocated to the Incubator for you.

######Is it possible to transfer GitHub issues from one project to the other? No, GitHub doesn’t allow it. (There are migration tools that can copy issues and their comments to another project, but these don't transfer the relevant author info; they assign ownership to the account used by the migration tool.)

######Is it possible to transfer the ownership of a Github repo to someone else? Yes.

####What Makes a Project InnerSource InnerSource projects meet all the same criteria as “coding in the open” projects, but with one major difference: they appear on GitHub Enterprise, and are not accessible to the public. No one outside of Zalando Tech can see or contribute to them.

Wait, I thought we were “Open Source First.” Doesn’t keeping repos on GHE contradict that?

Nope. Not every project we create will be appropriate for sharing publicly. Some projects will be too sensitive for publication. Other projects would act as “noise,” because they are too tightly coupled to what we do internally. An organization's open source footprint says a lot about that organization, especially if the org cannot maintain a good signal-to-noise ratio.

However, we still want you to share these projects inside Zalando. This is why we advocate the InnerSource collaboration model.

Put simply, InnerSource operates just like open-source in that project teams invite, accept and reject PRs; provide quality documentation; and build them gradually. The main difference is that InnerSource is limited to a single organization—in our case, Zalando Tech.

With InnerSource, we encourage you to make your GitHub Enterprise organization open to other internal teams so they can find out about your work, fix bugs, make PRs, and even add features that your own team currently has no time to develop.

We request that you use private GHE repositories only if they include sensitive information that can't store elsewhere.

Where InnerSource projects belong: GitHub Enterprise

#####How to InnerSource Here's our homegrown how-to with materials to get you started.

##Project Basics

###Code Review

Ask your team and other peers to:

  • review your code
  • install and
  • test your project prior to public release.

Not sure what to ask for, or how to peer-review? This list of 11 best practices should help.

###Creating a README

####Do:

####Don't:

  • Refer to Zalando specifics, such as internal teams and processes
  • Include large chunks of code without explaining what they represent
  • Include any code that presents security vulnerabilities

####Syntax and Formatting Markdown is the simplest and most easily understood syntax; we recommend using it for all your documentation. However, we realize that there are exceptions: PyPi, for example, uses reStructuredText, and the Python community in general doesn’t use Markdown. If Markdown isn’t practical, then we recommend using only one GitHub-supported markup format.

###Official Namespace

The official namespace for our projects is ‘org.zalando’, where applicable.

###Applying Changes

All repository changes, including those made by maintainers, should come from GitHub pull requests so that we can streamline review and change tracking (as per GitHub Flow). Everyone should have their own fork, though you can still edit READMEs/documentation/related files with the GitHub “Edit” button. The ‘master’ branch should be the accepted development head; pull requests get merged there.

###Versioning

Version all project artifacts should be semantically. Tag all versions in GitHub with the exact version name (like ‘0.1.0’; do not prefix tags with “v.” or similar). For a better user experience, use the GitHub “release notes” feature to add notes whenever you change something in the new version.

###Maintainers

Maintainers are the contact people for a project. They are also the only contributors who can package new versions and apply changes to the repository (i.e., merge pull requests). Every project must have at least one maintainer. This helper script gives us an overview of repos that users are maintainers for.

The Open Source Guild reserves the right to contact maintainers to ensure a project remains active/maintained. If the project is not being maintained, we will work with you to either find a new maintainer or remove the project from our organization page. Please be responsive to all internal queries about your project and its status.

####Create a Maintainers File

Every project needs a ‘MAINTAINERS’ file (listing all maintainers) at its root. Your build/packaging configuration file (e.g., pom.xml for Maven) can fulfill the purpose of a MAINTAINERS file. Format:

 [full name] <email address>
 [second full name] <email address>
 etc.

Our Catwatch application collects maintainers from the MAINTAINERS files.

####Be Prompt and Responsive

Respond promptly to pull requests and issues. “Within 72 hours” is a good window. Open issues do not make your project “look popular.” Instead, they make it look like you're neglecting your project. If project workload becomes unmanageable, ask the Guild or the community for help.

If you are away/on vacation and can’t respond to PRs/issues promptly, find someone who can.

If you're not going to accept a PR, reject it ASAP and include a brief explanation why.

If you're feeling maintainer burnout or facing some trolling behavior, give Jon Schinklert's Maintainers Guide to Staying Positive a read; it might help you to feel better.

####Use Issues Creatively Issues can be good for planning or for onboarding contributors. Issues should include a description of the point, question, discovery, or other detail prompting the issue.

Issues that consist solely of a title appear unprofessional and do not do much to invite discussion from the community.

Label your issues with clear tags. This is a great way to organize and categorize issues.

###Stay Secure #####Two-Factor Authentication Is Required GitHub organization members must enable Two-Factor Authentication (2FA/MFA) in keeping with our Open Source First principle to "Stay Secure." Read GitHub's post on 2FA for more information.

Don't have a smartphone, and/or want to give your phone number to GitHub? According to GitHub support, SMS or a TOTP app are currently the only primary 2FA methods that work. There are mobile and desktop TOTP apps that also work. See GitHub's article on 2FA via TOTP apps for more information. You can set up 2FA with a Google Voice SMS number, but should add a U2F device as backup.

###About Licensing

####Quick FAQ

#####Which license do we use? The MIT license. MIT is succinct, straightforward, and easy use in closed-source projects. It allows the most broad usage of our source code, and keeps open-sourcing easy and safe. You must include a separate license file in your repository with the entire text of the MIT license included.

If your project uses the MIT license and incorporates third-party OSS components, then you must attach this sentence in your license file: “This software is licensed under the MIT license (see below), unless otherwise stated in the license files in higher directories (if any)." Please also provide the license files for every third-party source you add.

This helpful blog post breaks down the license line by line for you. (If you're really interested in open source licensing, check out Rosenlaw & Einschlag's free online book and/or O'Reilly's/Andrew M. St. Laurent's. These are a little dated, but still useful.)

For documents like this how-to, we recommend using Creative Commons Attribution 4.0 International.

#####Must I use MIT? Yes, for all newly created open-source projects.

#####I don't like the MIT license. Can I use another license?

Not without a compelling reason.

#####What if I fork an external project and/or contribute to it?

Keep the original license.

#####I'm open-sourcing a library. What should I do? Consider using a weak copyleft license that won’t restrict the software that uses it to the same license; will allow usage in closed source software; and will potentially increase the number of users and contributors.

#####What if my team uses an external project whose license is not Zalando-recommended?

You can can use GPL code — but only internally. Be sure it's a version of the GPL that continues to allow for the ASP loophole. AGPL and versions of the GPL with additional restrictions won't work.

#####Who is the license owner?

Zalando SE.

#####Does my project need a LICENSE file?

Yes, at the root of the repository that contains the copy of the selected license (see above). Here is an example for MIT.

#####Do I need to add a license header to every source file?

No. In fact, we discourage it because it blows up file sizes, requires some build checking/pre-processing, and sometimes leads to situations like this. It's also not needed for the licenses we use. Having one LICENSE file in the repository is enough.

#####What about a README note?

Yep. Every README{.md,.rst} file must state the following at the end:

The MIT License (MIT) Copyright © [yyyy] Zalando SE, https://tech.zalando.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Replace the [yyyy] field with the year that you created the project, and do not update it. Do not provide multiple years.

#####I still have licensing questions. What can I do?

Ask the Open Source Guild or your delivery lead. See also our TechWiki page (internal doc) for a link to more detailed information.

#####But I am a delivery lead.

Ask your department head.

#####But I am a delivery head.

Management can work with Legal to determine Intellectual Property concerns.

####Repository of Meta Information

Many package managers include a feature to make the applied license machine readable. Use these! An example for Maven:

<licenses>
    <license>
        <name>MIT</name>
        <url>https://opensource.org/licenses/MIT</url>
        <distribution>repo</distribution>
    </license>
</licenses>

An example for Node, according to this:

“license”: “MIT”

An example for Scala (with sbt):

licenses += ("MIT", url("http://opensource.org/licenses/MIT"))

####Restrictions Imposed by the License “Dependency” typically means “being linked with,” “included in your artifact,” or “depends on it during runtime.” Dependencies can limit you. To remain in compliance, check the licenses of your projects. Your build tool’s license does not affect your software’s license. A jar file or Python dependency will affect your software.

#####Unusual Additions

As stated by Zalando Legal, it is OK to use React and other Facebook open-source software projects for Zalando projects.

####There Is No License

If there is no license statement, the author automatically receives a copyright. This implies that no one has the right to modify or redistribute the software. If you really need the software, contact the author (who is likely unaware) and ask him/her to provide a proper license.

###Other Repository Information

####JVM Artifacts Host JVM artifacts (*.jar) on Maven Central in the ‘org.zalando’ group. To do this, get a Sonatype account. If you don't have an account yet, register with Sonatype using your Zalando email address.

If you want to push to Sonatype but not to *.zalando, register a different Sonatype account with a non-Zalando email address.

####Python Packages Host Python packages on PyPI (PyPI has no namespaces) and make sure that multiple persons have “maintainer” rights.

####SBT plugins

####Node Modules Node modules now have namespaces. Prefix them with a short product name: e.g. karma plugins are prefixed “karma-”; the same goes for gulp, grunt, etc. Host your Node modules in the public NPM registry. Here is how to publish to NPM.

####Docker Images You can currently browse it here, or with the Pier One command line utility. We have an open source registry that everyone can read. It is deployed in the AWS open source account and Docker images can be pushed by any team member to their respective team repo:

$ # you need to login to registry-write.opensource.zalan.do (workaround for Docker V2 bug)
$ pierone login --url registry-write.opensource.zalan.do
$ docker push registry-write.opensource.zalan.do/myteam/myartifact:1.0
$ # on any other computer:
$ docker pull registry.opensource.zalan.do/myteam/myartifact:1.0 # no auth needed for download!

###Working with External Contributors

The Guild enthusiastically supports you in recruiting non-Zalandos to contribute to your project. Collaboration and community are part of the fun of open source.

Give GitHub's "Creating a new contributor on-ramp" doc a look for some expert guidance. (TL;DR version = be welcoming, inclusive, and clear about what sorts of contributions you're looking for most).

Please ask external contributors to sign a contributor licensing agreement (CLA). A few good models to follow and adapt: .Net Foundation example (electronic submission via GitHub account); Google’s CLA for contributing to AngularJS is a simple click-through form with a Googlebot that automatically checks for signatures; Selenium/Software Freedom Conservancy uses a Google form.

###Make Forks Meaningful

“Forks are for making your own snapshot of a codebase so that you can make a new version of it with your own special sauce, or so that you can contribute a change in the form of a pull request. Simply, you must make a fork whenever you need to modify the codebase, but do not have direct access to do so. New users don’t understand this and end up equating the ‘fork’ button with ‘download’ or ‘bookmark’. Little do they know, you can download code directly from the original repository and you can bookmark things using Github’s stars.” —Eric Greer

To fork, or not to fork? Some guidelines:

  • Only keep the fork open as long as needed.
  • “True/Long-Lived forks” are highly discouraged, even from Zalando projects. We avoid internal forks in order to avoid diverging widely from what we have published, and the inevitable hassle of getting the project to a state where we can support both code bases
  • Avoid two forks of the same project.

If your goal is to make a small fix to a project, use your own/personal GitHub account.

###Deprecate Responsibly

####How to Deprecate Add “Deprecated” at the top of your README, as well as a notice stating your plans for the project: deletion, finding a new owner, etc.

After announcing your decision to deprecate the project:

  • notify your users. Put a notice on your README.
  • Wait 60 days.
  • Try to find a new owner. Internal options first, then seek an external owner. Ask the Guild for help.
  • Try to find an external owner. More guidance on this to come soon; for now, alert the Guild of any such plans.

####Tips for Finding a New Owner Internally, you can use internal mailing lists and HipChat to announce your need. Externally, social media platforms and community boards work well. Add 1-2 sentences to your announcements suggesting how your project might have potential to evolve into something bigger and better.

###Project Promotion

####zalando.github.io: Our Open-Source Projects Dashboard All Zalando open-source projects are listed on zalando.github.io (also called CatWatch): our own metrics dashboard measuring our most popular public projects and our most active contributors in terms of commits, stars and forks. Please add a .catwatch.yaml file to the root of your repository to set a human-readable project title and image URL.

####Proactively Communicate Share your project with:

  • relevant LinkedIn Groups
  • community forums/boards
  • e-newsletters and websites/newsletters dedicated to the problem(s) your project is trying to solve, the relevant languages, etc.
  • if you have contacts at a university, pitch your project to their students
  • major players in the industry

###Contributing to Non-Zalando Open-Source Projects

We encourage you to contribute to other open-source projects in ways that benefit Zalando. Some ways you might contribute:

— making bug fixes in Apache

  • making bug fixes to projects you're using on the job
  • submitting a patch or change to a language (for example, Clojure)
  • pitching a feature request that your team/dept needs to a project we're using, getting confirmation from the maintainers to go forward, and doing the work
    • a good idea is to review the project's GitHub Issues Tracker to see if anyone else has made the same feature request; restart that conversation and see if you can get them + others to work collaboratively to make the needed change

####How to Become a Contributor

Before writing a line of code to change a project, we highly recommend reaching out to the project maintainers via the Issues Tracker and letting them know about your plans (many projects already require this as a first step). There might already be an issue filed that matches your needs, and/or someone working on the changes you're recommending. You don't ever want to invest a lot of time and work into making a change, only to have the project maintainers reject it as out of scope. This will frustrate you, and it will frustrate them.

These articles from others in the community provide more detailed advice for getting started:

####How to Become a Core Contributor

Ah, so you want to get more serious about contributing! Excellent. Becoming a core contributor to a major project is a fantastic way to learn from others, grow your network and keep the open source infrastructure we depend on strong and reliable. Others in the general open source community have written great how-to's on this topic. Here are a few:

Please let the Guild know about your external contributions so we can help you get the recognition and support you deserve.

#####CLAs

For typical CLAs, we are safe — but ask our legal team (guild can provide their contact info) to double-check whenever you’re in doubt. CLAs that are safe: Oracle, Apache, Google Projects.