The goal of the project is to test distributions from PyPI repository to assess
quality and also to check if a distribution is malicious or not . In order to
achieve that we create a testing infrastructure for PyPI repository. There will
be a mechanism to get newly uploaded distributions from PyPI , install them in
an isolated VM environment , run tests on them (quality check , unittests) and
also determine tests they have harmful components(malicious) or not. The
project can be divided into two parts , one(environment) to subscribe to
uploaded packages,set up the environment and the other one(execution) to run
the tests and report the results back(to the environment part).
This project can be divided into two components : one is execution part and
another is environment part . Since each of these two parts are comprehensible
enough on its own, each will be handled by a single student. Execution part
takes cares of installing the distributions(to be tested) along with their
dependencies , run tests on these distributions and assess different quality
parameters. Tests may include unittests, or quality tests(like pep8,pylint) or
custom tests to check if the program is malicious or not .
Environment part of the project
This proposal concerns the environment part. The environment part of the
project is responsible for creating an abstraction for the execution part . It
handles delivery of distributions (and its dependencies), to the execution part
( to run tests on them). It handles all the protocols required to communicate
to the PyPI repository and also to the different architecture used in the
project. It subscribes to uploaded packages from PyPI for testing them (testing
done by the execution part). It is also responisible for setting up the
environment required for testing and to deliver the packages to the execution
part for testing them .
- Raw data: the data generated by tasks execution.
- Report: evaluation of the different features/attributes of the data.
- Task : execution which produce raw data and "output". eg build, install,
- Master – Slave architecture where the master dispatches jobs to the slave and
the slave executes them.
- The communication between master and slave happen through an API called
- The slave communicates with the vm , sends the distributions require for testing
and receive raw data (after installing the distributions and conducting tests)
using another API called raw data API
- Tests are run on VMs and each VM is handled by a slave
Raw Data API
- The task is to build a raw data API for the communication between the VM and
- The raw data API handles sending the data into the corresponding VMs.
- The raw data API also handles the raw data (after the execution part has finished)
on VM to be sent to the slave
- The task is to build a command API to communicate between the Master and the
- The command API handles the task requests issued by the Master to and
assigns them to the slave.
- Task requests can involve different configurations to be made on a VM,
what distributions to be tested,etc.
- For both the API I propose the use of XML format or json format for creating the API as I
think both can be used easily and both have good support.
The slave performs the following tasks
- Initialises an isolated VM and configures the VM using the configuration
provided by the API call to it.
- It should be able to communicate with PyPI repository and get the
distributions to be tested.
- Gets the distribution to be tested from the repository , computes
dependencies and also gets the dependencies from the repository.
- Passes all the packages to the VM.
- Receive the raw data from the VM.
- The slave is required to differentiate between the different VMs and also
have a track of the activities happening with each of the VMs.
- The slave intializes and configures the VM by making an API call to it.
- When the packages are sent into the VM , they can be stored in a folder
and the execution part can keep polling into the folder to see if any package
has been received to start testing.
- Master subscribes to uploaded packages in PyPI
- It dispatches jobs to the Slave using command API
- It receives the test results from the slave
- Inorder to subscribe packages from PyPI , we can use pubhubsubbub protocol to
get real time feed as and when a package has been uploaded.
- Developer uploads his distribution on pypi. (External)
- Pypi notifies PYTI(Here the master gets notified). (Environment)
- Master asks a slave(local or remote machine) to test a distribution
using command API.(Environment)
- Slave computes the dependencies for the distribution(to be tested) and
downloads the distribution(to be tested) along with the dependencies from
the repository. (Execution)
- Slave starts a VM with settings as instructed by the Master (Environment)
- When the VM has started,the slave sends distribution(along with the
dependencies) into the VM.(Environment)
- Inside the VM , the distribution is installed and different tests are
conducted on it (unittests, quality check ,etc) (Execution)
- At the end, raw data(data obtained by testing) is sent to slave. (Execution)
- Slave sends raw data to master. (Environment)
- Then slave shutdown the VM and cleans it. (Environment)