Exploring inter-services communication using AQMP brokers like RabbitMQ among python and java based microservices.
cd into each of the front-end, backend services directory and initiate them:
vagrant up --provider virtualbox
java-api-backend$ gradle run
java-api-backend$ cd ../knockout-frontend
knockout-frontend$ python -m SimpleHTTPServer 8090
knockout-frontend$ cd ../python-scraping-service
python-scraping-service$ pip install -r requirements.txt
python-scraping-service$ python worker.py
Note: You must have wget and apt-get installed on Mac OS X for the vagrant scripts to execute successfully.
To install wget, simply run: brew install wget
I've used virtualbox with ubuntu/trusty64 environment for running this. If you don't have it setup, the easiest to do this in Mac OS X is using Homebrew Cask:
$ brew cask install virtualbox
$ brew cask install vagrant
Vagrant Manager helps you manage all your VMs in one place from menubar
$ brew cask install vagrant-manager
Every Vagrant development environment requires a box. You can search for boxes at link
The virtual box used for this demo is ubuntu/trusty64 link
$ vagrant init ubuntu/trusty64;
You should get a CL message: A Vagrantfile has been placed in this directory. You are now ready to vagrant up your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
vagrantup.com for more information on using Vagrant.
$ vagrant up --provider virtualbox
To get rid of the vagrant VM, just run vagrant destroy
Also, you may find it easier to manage the virtual boxes using vagrant-manager, which can be installed using brew install vagrant-manager
Note: to make things easier, in application.properties the line spring.jpa.hibernate.ddl-auto=create allows us to automatically create the tables when starting up. However, this means all tables get erased with each new start. Change the line to spring.jpa.hibernate.ddl-auto=validate (after you have started up the backend at least once) to avoid data loss.
Pika link: Pika is a pure-Python implementation of the AMQP 0-9-1 protocol that tries to stay fairly independent of the underlying network support library.
For the actual scraping of a website’s summary, we use https://github.com/raeidsaqur/sumy library. Please see their docs (especially as it requires to have nltk installed on your system).
The scraper is not doing much except using the sumy library and returning the result.
To run the python scraper, install requirements.txt (preferably in a virtualenv) and then simply run worker.py:
(venv)python-scraping-service$ python worker.py
Spring has a boot starter module available for RabbitMQ link org.springframework.boot:spring-boot-starter-amqp, which we add to our dependencies.