Note: Obsei is still in alpha stage hence carefully use it in Production. Also, as it is constantly undergoing development hence master branch may contain many breaking changes. Please use released version.
Obsei (pronounced "Ob see" | /əb-'sē/) is an open-source, low-code, AI powered automation tool. Obsei consists of -
- Observer: Collect unstructured data from various sources like tweets from Twitter, Subreddit comments on Reddit, page post's comments from Facebook, App Stores reviews, Google reviews, Amazon reviews, News, Website, etc.
- Analyzer: Analyze unstructured data collected with various AI tasks like classification, sentiment analysis, translation, PII, etc.
- Informer: Send analyzed data to various destinations like ticketing platforms, data storage, dataframe, etc so that the user can take further actions and perform analysis on the data.
All the Observers can store their state in databases (Sqlite, Postgres, MySQL, etc.), making Obsei suitable for scheduled jobs or serverless applications.
- Text, Image, Audio, Documents and Video oriented workflows
- Collect data from every possible private and public channels
- Add every possible workflow to an AI downstream application to automate manual cognitive workflows
Obsei use cases are following, but not limited to -
- Social listening: Listening about social media posts, comments, customer feedback, etc.
- Alerting/Notification: To get auto-alerts for events such as customer complaints, qualified sales leads, etc.
- Automatic customer issue creation based on customer complaints on Social Media, Email, etc.
- Automatic assignment of proper tags to tickets based content of customer complaint for example login issue, sign up issue, delivery issue, etc.
- Extraction of deeper insight from feedbacks on various platforms
- Market research
- Creation of dataset for various AI tasks
- Many more based on creativity đź’ˇ
Here are some companies/projects (alphabetical order) using Obsei. To add your company/project to the list, please raise a PR or contact us via email.
- 1Page: Giving a better context in meetings and calls
- Spacepulse: The operating system for spaces
- Superblog: A blazing fast alternative to WordPress and Medium
- Zolve: Creating a financial world beyond borders
| Sr. No. | Workflow | Colab | Binder |
|---|---|---|---|
| 1 | Observe app reviews from Google play store, Analyze them by performing text classification and then Inform them on console via logger | ||
| PlayStore Reviews → Classification → Logger |
|
|
|
| 2 | Observe app reviews from Google play store, PreProcess text via various text cleaning functions, Analyze them by performing text classification, Inform them to Pandas DataFrame and store resultant CSV to Google Drive | ||
| PlayStore Reviews → PreProcessing → Classification → Pandas DataFrame → CSV in Google Drive |
|
|
|
| 3 | Observe app reviews from Apple app store, PreProcess text via various text cleaning function, Analyze them by performing text classification, Inform them to Pandas DataFrame and store resultant CSV to Google Drive | ||
| AppStore Reviews → PreProcessing → Classification → Pandas DataFrame → CSV in Google Drive |
|
|
|
| 4 | Observe news article from Google news, PreProcess text via various text cleaning function, Analyze them via performing text classification while splitting text in small chunks and later computing final inference using given formula | ||
| Google News → Text Cleaner → Text Splitter → Classification → Inference Aggregator |
|
|
|
đź’ˇTips: Handle large text classification via Obsei
We have a minimal streamlit based UI that you can use to test Obsei.
Check demo at
(Note: Sometimes the Streamlit demo might not work due to rate limiting, use the docker image (locally) in such cases.)
To test locally, just run
docker run -d --name obesi-ui -p 8501:8501 obsei/obsei-ui-demo
# You can find the UI at http://localhost:8501
To run Obsei workflow easily using GitHub Actions (no sign ups and cloud hosting required), refer to this repo.
For detailed installation instructions, usages and examples, refer to our documentation.
| Linux | Mac | Windows1 | Remark | |
|---|---|---|---|---|
| Tests | âś… | âś… | âś… | Low Coverage as difficult to test 3rd party libs |
| PIP | âś… | âś… | âś… | Fully Supported |
| Conda2 | âś… | âś… | âś… | Partially Supported |
1. On Windows you have to install pytorch manually. Refer to the Pytorch official instruction. ↩
2. Conda channel is missing few dependencies, hence install missing dependencies manually - ↩
Missing Conda dependencies -
pip install presidio-analyzer
pip install presidio-anonymizer
pip install zenpy
pip install searchtweets-v2
pip install google-play-scraper
pip install tweet-preprocessor
pip install gnews
pip install python-facebook-api
# GPL dependency
pip install trafilaturaExpand the following steps and create your workflow -
Step 1: Prerequisite
Install the following (if not present already) -
- Install Python 3.7+
- Install PIP (Optional if you prefer Conda)
- Install Conda (Optional if you prefer PIP)
Step 2: Install Obsei
You can install Obsei either via PIP or Conda based on your preference.
NOTE: On Windows you have to install pytorch manually. Refer to https://pytorch.org/get-started/locally/.
To install latest released version -
pip install obseiInstall from master branch (if you want to try the latest features) -
git clone https://github.com/obsei/obsei.git
cd obsei
pip install --editable .To install the latest version -
conda install -c lalitpagaria obseiInstall from master branch (if you want to try the latest features) -
git clone https://github.com/obsei/obsei.git
cd obsei
conda env create -f conda/environment.ymlFor GPU based local environment -
git clone https://github.com/obsei/obsei.git
cd obsei
conda env create -f conda/gpu-environment.ymlStep 3: Configure Source/Observer
|
|
|
|
|
|
|
|
|
|
|
|
Step 4: Configure Analyzer
Note: To run transformers in an offline mode, check transformers offline mode.
Some analyzer support GPU and to utilize pass device parameter. List of possible values of device parameter (default value auto):
- auto: GPU (cuda:0) will be used if available otherwise CPU will be used
- cpu: CPU will be used
- cuda:{id} - GPU will be used with provided CUDA device id
|
|
|
|
|
|
Step 5: Configure Sink/Informer
|
|
|
|
|
|
|
Step 6: Join and create workflow
source will fetch data from the selected source, then feed it to the analyzer for processing, whose output we feed into a sink to get notified at that sink.
# Uncomment if you want logger
# import logging
# import sys
# logger = logging.getLogger(__name__)
# logging.basicConfig(stream=sys.stdout, level=logging.INFO)
# This will fetch information from configured source ie twitter, app store etc
source_response_list = source.lookup(source_config)
# Uncomment if you want to log source response
# for idx, source_response in enumerate(source_response_list):
# logger.info(f"source_response#'{idx}'='{source_response.__dict__}'")
# This will execute analyzer (Sentiment, classification etc) on source data with provided analyzer_config
analyzer_response_list = text_analyzer.analyze_input(
source_response_list=source_response_list,
analyzer_config=analyzer_config
)
# Uncomment if you want to log analyzer response
# for idx, an_response in enumerate(analyzer_response_list):
# logger.info(f"analyzer_response#'{idx}'='{an_response.__dict__}'")
# Analyzer output added to segmented_data
# Uncomment to log it
# for idx, an_response in enumerate(analyzer_response_list):
# logger.info(f"analyzed_data#'{idx}'='{an_response.segmented_data.__dict__}'")
# This will send analyzed output to configure sink ie Slack, Zendesk etc
sink_response_list = sink.send_data(analyzer_response_list, sink_config)
# Uncomment if you want to log sink response
# for sink_response in sink_response_list:
# if sink_response is not None:
# logger.info(f"sink_response='{sink_response}'")Step 7: Execute workflow
Copy the code snippets from Steps 3 to 6 into a python file, for exampleexample.py and execute the following command -
python example.py| Sr. No. | Title | Author |
|---|---|---|
| 1 | AI based Comparative Customer Feedback Analysis Using Obsei | Reena Bapna |
| 2 | LinkedIn App - User Feedback Analysis | Himanshu Sharma |
Discussion about Obsei can be done at community forum
Upcoming release plan and progress can be tracked at link (Suggestions are welcome). Refer releases for changelogs.
For any security issue please contact us via email
This project is being maintained by Lalit Pagaria.
- Copyright holder: Lalit Pagaria
- Overall Apache 2.0 and you can read License file.
- Multiple other secondary permissive or weak copyleft licenses (LGPL, MIT, BSD etc.) for third-party components refer Attribution.
- To make project more commercial friendly, we void third party components which have strong copyleft licenses (GPL, AGPL etc.) into the project.
This could not have been possible without these open source softwares.
First off, thank you for even considering contributing to this package, every contribution big or small is greatly appreciated. Please refer our Contribution Guideline and Code of Conduct.
Thanks so much to all our contributors