Out Of Core Numpy Python Developer Internship

Out Of Core Numpy Python Developer

Description:
In 2014 Nexedi has created a technology called Wendelin.core which provides out-of-core python ndarrays that can be shared transparently across different nodes of cluster of python runtimes. With Wendelin.core, python can be used natively for big data without relying on other languages or runtimes. Wendelin.core is used in production for example to monitor wind turbines and detect anomalies.

Nexedi is looking for a trainee interested in making improving libraries utilized by our Wendelin and wendelin.core, improving the source code of NumPy and scikit-learn to reduce the number of memory allocations or copies. This task may require to modify default algorithms that use array allocations with algorithms that modify data in-place. It may also require to allocate explicitly out-of-core ndarrays whenever there is no better way. Overall, both NumPy and scikit-learn will be improved by relying as little as possible on array allocations.

Opportunities:

• Contribute to NumPy and scikit-learn.
• Improve Wendelins and wendelin.core’s capabilities to analyse TB of data.

Responsibilities:

• Learn to setup and use Wendelin, the Jupyter extensions and libraries.
• Learn how to write memory-aware data handlers.
• Learn how to contribute to popular open source libraries.

Requirements:

• Passionate, self-driven.
• Willingness to contribute to an open source ecosystem and the Free Software community.
• Very good programming skills in Python.
• Good skills in algorithms and experience with NumPy, scikit-learn.
• Good software development skills (version control, testing, debugging).