Skip to main content

Creating Tailored Research Computing Environments

Project Information

big data, cpus, gpus, parallelization, resources, containers, docker, Machine-Learning, r, python, ,
Project Status: In Progress
Submitted By: David Oury
Project Email:
Project Institution: Bentley University
Anchor Institution: MGHPCC
Project Address: 175 Forest Street
Waltham, Massachusetts. 02452

Mentors: David Oury
Students: Snehal Mattoo

Project Description

The cloud (public and private) provides an array of virtual machines, available with a range of cores, RAM and specialized hardware. Research faculty at small and medium-size institutions have a variety of requirements for computing and data resources, but need to be efficient in their use of these resources. Three current best practices (Terraform, Docker, AWS S3) are an integrated toolset able to provision compute/data resources and configure compute/data services in a customizable and efficient manner. The primary objective of the Tailored Research Environments (directed study) project is to create a web-based product that enables its user to run R, Python and Spark code in Jupyter notebooks on a user configured computer environment. Importantly, the user's notebooks (code) and datasets are stored separately from the computing environment. In addition, compute resources (of this environment) can be added and removed as needed, which will significantly reduce the overall cost of using the product.