Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: claimed-framework/component-library
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.2rc0
Choose a base ref
...
head repository: claimed-framework/component-library
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Nov 30, 2022

  1. add vulnerabilities process

    Romeo Kienzler committed Nov 30, 2022
    Copy the full SHA
    2bec952 View commit details

Commits on Dec 1, 2022

  1. add automated tests

    Romeo Kienzler committed Dec 1, 2022
    Copy the full SHA
    c5fc7d2 View commit details

Commits on Dec 2, 2022

  1. Copy the full SHA
    79db32a View commit details
  2. Copy the full SHA
    4399e75 View commit details
  3. Copy the full SHA
    7f24515 View commit details
  4. Copy the full SHA
    ce7e2bf View commit details
  5. fix requirements

    romeokienzler authored Dec 2, 2022
    Copy the full SHA
    21adb39 View commit details
  6. fix path

    romeokienzler authored Dec 2, 2022
    Copy the full SHA
    6dd535f View commit details
  7. fix path

    romeokienzler authored Dec 2, 2022
    Copy the full SHA
    d45f02f View commit details
  8. fix path

    romeokienzler authored Dec 2, 2022
    Copy the full SHA
    19f8762 View commit details
  9. Copy the full SHA
    8f90663 View commit details
  10. Copy the full SHA
    9d28f65 View commit details
  11. Copy the full SHA
    2af23a3 View commit details

Commits on Dec 5, 2022

  1. Copy the full SHA
    93a556d View commit details

Commits on Dec 7, 2022

  1. rename test

    romeokienzler authored Dec 7, 2022
    Copy the full SHA
    a9ee799 View commit details
  2. rename test

    romeokienzler authored Dec 7, 2022
    Copy the full SHA
    8985a81 View commit details
  3. rename test

    romeokienzler authored Dec 7, 2022
    Copy the full SHA
    cf38e66 View commit details
  4. rename action

    romeokienzler authored Dec 7, 2022
    Copy the full SHA
    3a57766 View commit details
  5. Corrected typo

    apoorvarajlondhe committed Dec 7, 2022
    Copy the full SHA
    90f83e0 View commit details

Commits on Dec 8, 2022

  1. Merge pull request #212 from apoorvarajlondhe/master

    Corrected some minor typo
    romeokienzler authored Dec 8, 2022
    Copy the full SHA
    4168adf View commit details
  2. Copy the full SHA
    a194359 View commit details
  3. Added LICENSE file

    ibrahimhaddad authored Dec 8, 2022
    Copy the full SHA
    b30811c View commit details
  4. Delete LICENSE.md

    ibrahimhaddad authored Dec 8, 2022
    Copy the full SHA
    c0f0305 View commit details
  5. Copy the full SHA
    cd89e3b View commit details

Commits on Dec 9, 2022

  1. add changelog

    romeokienzler authored Dec 9, 2022
    Copy the full SHA
    e9a2d01 View commit details
  2. add SECURITY.md

    romeokienzler authored Dec 9, 2022
    Copy the full SHA
    6c8b167 View commit details
  3. add SUPPORT.md

    romeokienzler authored Dec 9, 2022
    Copy the full SHA
    68e9187 View commit details
  4. Copy the full SHA
    1c1cc88 View commit details
  5. add PR template

    Romeo Kienzler committed Dec 9, 2022
    Copy the full SHA
    346d990 View commit details
  6. add CODEOWNERS

    romeokienzler authored Dec 9, 2022
    Copy the full SHA
    73488d9 View commit details
  7. fix names

    romeokienzler authored Dec 9, 2022
    Copy the full SHA
    b1278f1 View commit details
  8. fix code owners

    romeokienzler authored Dec 9, 2022
    Copy the full SHA
    c1ed729 View commit details

Commits on Jan 11, 2023

  1. Update README.md

    romeokienzler authored Jan 11, 2023
    Copy the full SHA
    5e34880 View commit details

Commits on Jan 13, 2023

  1. Create FAQ.md

    romeokienzler authored Jan 13, 2023
    Copy the full SHA
    1377a25 View commit details
  2. link FAQ.md

    romeokienzler authored Jan 13, 2023
    Copy the full SHA
    25ddd80 View commit details

Commits on Jan 24, 2023

  1. Add files via upload

    romeokienzler authored Jan 24, 2023
    Copy the full SHA
    a086f72 View commit details

Commits on Feb 10, 2023

  1. add gdal component

    romeokienzler committed Feb 10, 2023
    Copy the full SHA
    0ac87d8 View commit details
  2. Copy the full SHA
    75ae238 View commit details

Commits on Mar 14, 2023

  1. add updates

    romeokienzler committed Mar 14, 2023
    Copy the full SHA
    44cb6e9 View commit details
  2. remove unneccesary file

    romeokienzler committed Mar 14, 2023
    Copy the full SHA
    f7c8d54 View commit details
  3. Merge pull request #218 from romeokienzler/cleanup

    remove unneccesary file
    romeokienzler authored Mar 14, 2023
    Copy the full SHA
    e92b55f View commit details
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/-report-a-vulnerability-.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: " Report a vulnerability "
about: " Report a vulnerability "
title: ''
labels: ''
assignees: ''

---

### Summary
_Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server._

### Details
_Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer._

### PoC
_Complete instructions, including specific configuration details, to reproduce the vulnerability._

### Impact
_What kind of vulnerability is it? Who is impacted?_
22 changes: 22 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.

Please note that additional functionality must be covered by automated testing: https://github.com/claimed-framework/community/blob/main/contribution_process.md
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/issue-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: Issue report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the issue**
A clear and concise description of what the issue is.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Screenshots or log output**
If applicable, add screenshots or log output to help explain your problem.
<details><summary>Log Output</summary>
<pre>
Paste the log output here.
</pre>
</details>

**Expected behavior**
A clear and concise description of what you expected to happen.

**Deployment information**
Describe what you've deployed and how:
- CLAIMED version: [e.g. 1.5.3]
- Installation source: [e.g. git, dockerhub]
- Runtime type: [e.g. ipython, docker, kubernetes, knative, kubeflow ]
44 changes: 44 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor code that leads to class changes, showing the updated class hierarchy will help reviewers.
2. If there is design documentation, please add the link.
-->

### How was this pull request tested?
<!--
Please make sure to add some test cases that check the changes thoroughly
including negative and positive cases, if possible.
If changes were tested in a way different from regular unit tests, please clarify how you tested step by step,
ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added e.g. documentation only, GH workflow updates, etc.
-->


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the Apache License 2.0; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
2 changes: 1 addition & 1 deletion .github/workflows/build_push_container_image.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: build-push-container-image
name: Build and Push Container Images

on:
push:
33 changes: 33 additions & 0 deletions .github/workflows/integration-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Integration Tests

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

permissions:
contents: read

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r test_requirements.txt
- name: Full integration test
run: |
cd component-library
python ./run_tests.py
24 changes: 24 additions & 0 deletions .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Static Code Analysis (Pylint)

on: [push]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10"]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pylint
- name: Analysing the code with pylint
run: |
pylint $(git ls-files '*.py')
pylint $(git ls-files '*.ipynb')
33 changes: 33 additions & 0 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Dynamic Code Analysis

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

permissions:
contents: read

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install crosshair-tool
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Test with pytest
run: |
crosshair watch ./component-library
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Changelog

- See our changelog in the release section [here](https://github.com/claimed-framework/component-library/releases)
2 changes: 2 additions & 0 deletions CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@romeokienzler
@fatkaratekid
54 changes: 54 additions & 0 deletions FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# What is the CLAIMED Library?

CLAIMED is a library of re-usable coarse-grained data processing
components to create Data & AI pipelines without programming skills

# How does the CLAIMED Library help developers and data scientists?

Lead Data Scientists and Domain Experts contribute to the library to
create opinionated, tested and re-usable components which are consumed
by citizen data scientists and developers which enables them to create
state of the art Data & AI workflows

# Why did IBM decide to contribute this open source project to LFAI?

An open source project is only as strong as its community. IBM wants to
grow the community around CLAIMED and Elyra since both projects are
very strategic open source projects for IBM and RedHat

# When did IBM open source it?

The initial repository was created in 2015 and was originally used to
support the online courses IBM provides on Coursera.org and EDX.org.
From the very positive feedback IBM received from the learners, we
decided to create a general purpose library for AI, Machine Learning,
ETL and Data Science.

# Is there any competing project at IBM, or outside of IBM?

No. Open Source and open standards are the key principles of CLAIMED.
Therefore CLAIMED can be used in various contexts and therefore doesn't
compete but integrate.

# What action do we want the open source community to take?

We are actively looking for developers and data scientists to use the
library for the daily work including production ready software. We also
want them to report issues, fix issues via pull request, participate in
our discussions and contribute new components to the library


# How do we want you to use CLAIMED?

CLAIMED can be consumed in many ways and we encourage and support all
scenarios. As each CLAIMED component is backed by a jupyter notebook or
(R|python|bash) script and defines a clear interface they can be
invoked directly from source code or from a command line. This way a
Data & AI pipeline can be build by arbitrary code or shell scripts. The
next level is using docker. As each CLAIMED component is automatically
compiled into a container image, a set of "docker run" commands will do
the job. Finally, CLAIMED also creates Kubeflow Pipeline Component
specifications automatically, therefore, CLAIMED can be used in any
Kubeflow Pipeline setting, where the Gold standard is using ML Exchange
as component repository and Elyra as graphical pipeline editor.

File renamed without changes.
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
[![OpenSSF Best Practices](https://bestpractices.coreinfrastructure.org/projects/6718/badge)](https://bestpractices.coreinfrastructure.org/projects/6718)
[![GitHub](https://img.shields.io/badge/issue_tracking-github-blue.svg)](https://github.com/claimed-framework/component-library/issues)



# CLAIMED - The Component Library for AI, Machine Learning, ETL, and Data Science

**TL;DR**
- set of re-usable coarse-grained components (just a bunch of code)
- think of tasks, not functions (e.g., read from a database, transform data, train model, deploy model)
- think of tasks, not functions (e.g., read from a database, transform data, train model, deploy model, store result to cloud object storage)
- write once, runs everywhere: Kubeflow, Apache Airflow, CLI, KNative, Docker, Kubernetes
- orchestrate with anything: shell script, Kubeflow, Airflow, Argo, Tekton
- persistence layer / queue agnostic: Cloud Object Storage, file systems, PVC, Kafka, MQTT
- just use Python - no other skills required (no Kubeflow component YAML, maven, Java)
- 1st class citizen in JupyterLab and the Elyra Pipeline Editor (creating a low code / no code IDE for data science)
- upstream repository to IBM Watson Studio Pipelines contributed components in IBM Cloud Pak for Data


You also might wanna have a look at our [FAQ](FAQ.md)


CLAIMED is a component library for artificial intelligence, machine learning, "extract, transform, load" processes, and data science. The goal is to enable low-code/no-code rapid prototyping style programming to seamlessly CI/CD into production. The library provides ready-made components for various business domains, supports multiple computer languages, works on different data flow editors and command line tools, and runs on various execution engines including Kubernetes, KNative, Kubeflow, Airflow or plain docker. To demonstrate its utility, we constructed a workflow composed exclusively of this library's components. To display the capabilities of this library, we made use of a publicly available Computed Tomography (CT) scan dataset [covidata]. We created a deep learning model, which is supposed to classify exams as either COVID-19 positive or negative. We built the pipeline with Elyra's Pipeline Visual Editor, with support for local, Airflow, and Kubeflow execution [https://arxiv.org/abs/2103.03281](https://arxiv.org/abs/2103.03281).
@@ -62,8 +67,12 @@ To learn more on how this library works in practice, please have a look at the f

## Getting Help

We welcome your questions, ideas, and feedback. Please create an [issue](https://github.com/IBM/claimed/issues) or a [discussion thread](https://github.com/IBM/claimed/discussions).
We welcome your questions, ideas, and feedback. Please create an [issue](https://github.com/claimed-framework/component-library/issues) or a [discussion thread](https://github.com/claimed-framework/component-library/discussions).
Please see [VULNERABILITIES.md](VULNERABILITIES.md) for reporting vulnerabilities.

## Contributing to CLAIMED
Interested in helping make CLAIMED better? We encourage you to take a look at our
[Contributing](CONTRIBUTING.md) page.

## License
This software is released under Apache License v2.0
1 change: 1 addition & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Please see the following [file](https://github.com/claimed-framework/component-library/blob/master/VULNERABILITIES.md) for SECURITY related content
7 changes: 7 additions & 0 deletions SUPPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@



# Support

For support, please contact us on the [CLAIMED workspace](https://matrix.to/#/!HxmUqvBKioTlNGyZGJ:matrix.org?via=matrix.org) on Matrix.org, please use the [Issue Tracker](https://github.com/claimed-framework/component-library/issues)
if you feel that you've found a defect or want to propose a new feature.
7 changes: 7 additions & 0 deletions VULNERABILITIES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# VULNERABILITIES reporting process

Vulnerabilities can be reported on the GitHub Issue Tracker but reporting them on a secure and private channel is preferred.

The current way of reporting private vulnerabilities is using https://encrypt.to/romeokienzler.

We are working on a more generic channel and update this information accordingly
Loading