Job details « Go back to category
Full-time Python Out Of Core Numpy DeveloperPublished at 04.02.2020 - Viewed: 568 times - Nexedi SA in Lille, France
Task : Python Out-Of-Core Numpy Developer
Location : Lille, France or Munich, Germany
Type : Job/Trainee
Duration : Permanent/4-6 Months
Reference : Offer-2020-Numpy-Developer
Description : In 2014 Nexedi developed a technology called Wendelin.core which provides out-of-core python ndarrays that can be shared transparently across different nodes of a cluster of Python runtimes. With Wendelin.core, Python can be used natively for big data without relying on other languages or runtimes. Wendelin.core is already being used in production for example for monitoring offshore wind turbines and detecting anomalies. As most use cases of Wendelin.core involve third-party libraries such as NumPy or scikit-learn who run methods “not aware” of available memory, a key challenge for us is to ensure the libraries we deploy in production perform under heavy data loads.
Nexedi is looking for a candidate interested in improving libraries utilized by Wendelin and wendelin.core (mostly NumPy, scikit-learn to some degree, other depending on specific implementation) to reduce the number of memory allocations or copies made internally. This task may require to modify default algorithms that use array allocations or replace them with algorithms that modify data in-place. It may also require to allocate explicitly out-of-core ndarrays whenever there is no better way and contribute any changes made back to upstream improving libraries utilized for the community.
- Contribute to NumPy, scikit-learn and other libraries used in Wendelin implementations.
- Improve Wendelins and wendelin.core’s capabilities to analyse TB of data.
- Help harden our stack of free software solutions.
- Learn to master and improve Wendelin, Jupyter extensions and libraries.
- Contribute memory-aware data handlers to popular Python libraries.
- Add your knowhow and experience to industrial implementations.
- Passionate, self-driven.
- Willingness to contribute to an open source ecosystem and the Free Software community.
- Very good programming skills in Python.
- Very good skills in algorithms and experience with NumPy, scikit-learn.
- Good software development skills (version control, testing, debugging).
- Good command of English.
Nexedi has been developing free software since launching in 2001. We are maintaining software solutions (see our full stack) with over 10 million lines of code including:
- ERP5 – ERP/CRM/DMS/e-business
- Slapos – Cloud Orchestration and deployment
- Wendelin – Big Data/Machine Learning
- Neo – Distributed Storage
- Resist – Resilient Mesh Network
- Renderjs – Promise based component framework
- jIO – Virtual File System and storage connector
- NayuOS – Private OS
- OfficeJs – Private, offline capable productivity AppStore
Besides participating in various research initiatives, Nexedi provides customisation services for solutions with implementations being used by corporations such as Airbus, Sanef, Mitsubishi all around the world. We follow the principles of reflexive programming, enforce strict unit testing and emphasise using the latest web technologies.
Our economic model requires each developer to fulfill R&D objectives aiming at delivering short to medium sized solutions to customer requirements and progressing the evolution of our software stack. This way we try to be innovative and fund long term free software without the need for venture capital.