Research Software Curriculum

Research Software Curriculum

These materials represent a curated curriculum designed to help you develop and maintain your repository and code base. All of the resources listed below are free to access and use, and supplementary material like video lessons has been added where possible and relevant.

Bash

• What is Bash? - of all the shells available, Bash is one of the most popular, the most powerful, and the most friendly
• Bash Essentials – Bash commands commonly used to navigate file directories in your Terminal/GitBash
• The Unix Shell lesson from Software Carpentries - use of the shell is fundamental to using a wide range of other powerful tools and computing resources. These lessons will start you on a path towards using these resources effectively
• Installation Instructions - particularly important for Windows users, as Bash comes pre-installed on Mac. Windows users will need to install GitBash following the instructions at this link
• Using the Terminal in Mac - The Terminal app allows you to control your Mac using a command prompt

Git

• What is Git?- 2-minute video overview of the technology and how it works
• Install Git from GitHub Guides - Check if Git is already installed on your machine; if not, follow these instructions to get started. Notes: If you’ve already installed GitBash on Windows OS, you will have Git already. Installing GitHub Desktop will also install the latest version of Git if you don’t already have it.
• Installing Git from Software Carpentries - Alternative installation instructions from Software Carpentries, including videos and details per OS.
• Intro to version control with Git from Code Refinery – self-paced introductory lesson to version control using Git
• Git Intro video lesson from Code Refinery - Day 1 - Recorded lesson from a May 2021 Code Refinery workshop on material in Intro to version control with Git, part 1/2
• Git Intro video lesson from Code Refinery - Day 2 - Recorded lesson from a May 2021 Code Refinery workshop on material in Intro to version control with Git, part 2/2
• Branching and merging – lesson from Code Refinery on concept of branching in Git (featuring octopus diagram)
• What is .gitignore? – introduction to how and why to use the .gitignore file to not track some files in a project folder (e.g., because of their size or sensitivity)
• Git command cheat sheet – commonly used Git commands in one page that can also be downloaded

GitHub/GitLab (remote repositories)

• Understanding the GitHub flow – guide from GitHub on how and why to work with branches
• Collaborative distributed version control - We have learned how to make a git repository for a single person. What about sharing?
• SSH connection to GitHub – instructions to set up SSH connection to GitHub so that you do not need to input your login credentials with every push/pull
• Gitlab and SSH keys - instructions to add an SSH key to your (TU Delft) GitLab account for the same reason as above
• GitHub without the Command Line from Code Refinery - practice collaborating and sharing using either the GitHub website or GitHub desktop application
• GitHub Guides: Mastering Markdown - Markdown is a lightweight and easy-to-use syntax for styling all forms of writing on the GitHub platform.

Jupyter Notebooks and JupyterLab

• Introduction to Jupyter and JupyterLab - lesson material on the user interface of JupyterLab, how Jupyter notebooks work, and what some common and powerful usecases are

Anaconda Navigator and managing conda environments

• Anaconda Installation Guide from Software Carpentries - Although one can install a plain-vanilla Python and all required libraries by hand, we recommend installing Anaconda, a Python distribution that comes with the latest version of Python and Jupyter Notebooks by default
• Intro to Anaconda Navigator - Anaconda Navigator is a graphical user interface to the conda package and environment manager. This 10-minute guide to Navigator will have you navigating the powerful conda program in a web-like interface without having to learn command line commands
• Introduction to Conda for (Data) Scientists - Conda is an open source package and environment management system that easily creates, saves, loads, and switches between environments on your local computer
• Managing Conda environments - documentation on performing a range of common tasks with Conda using the command line

Python

• Scientific Computing with Python - a free video course series that teaches the basics of using Python 3
• Applied Data Science with Python Specialization - Coursera course in which you can enroll for free
• LearnPython.org - Whether you are an experienced programmer or not, this website is intended for everyone who wishes to learn the Python programming language
• Programming with Python from Software Carpentries - this introduction to Python is built around a common scientific task: data analysis
• Plotting and Programming with Python from Software Carpentries - an introduction to programming in Python for people with little or no previous programming experience using plotting as its motivating example
• Data Analysis and Visualization with Python for Social Scientists - basic information about Python syntax, the Jupyter notebook interface, how to import CSV files, using the pandas package to work with data frames, how to calculate summary information from a data frame, and a brief introduction to plotting. The last lesson demonstrates how to work with databases directly from Python
• Can You Speak Python? - test your knowledge of some important features of the Python programming language and the NumPy and Pandas libraries

Pandas

• Getting started with Pandas - documentation and quick start guide for Pandas, an essential Python library used for working with data sets. It has functions for analyzing, cleaning, exploring, and manipulating data
• Pandas Tutorial - 14-part tutorial series featuring live code examples and tests of your knowledge
• Pandas Data Wrangling Cheat Sheet - to excel data analysis/data science/machine learning in Python, Pandas is a library you need to master. Here is a cheat sheet of some of the most used syntax that you probably don’t want to miss
• Pandas Cheat Sheet - Visual - visual, printable 2-page reference guide on commonly performed operations using Pandas
• Ultimate Pandas Guide — Inspecting Data Like a Pro - Whether you’re working on a simple analysis or a complex machine learning model, there’s a lot of value in being able to answer quick, exploratory questions about the nature of your data. This is a walk through of several DataFrame attributes and methods that make data inspection painless and productive
• 10 Efficient Ways for Inspecting a Pandas DataFrame Object - A guide to using pandas effectively and efficiently

Plotly

• Getting Started with Plotly - The plotly Python library is an interactive, open-source plotting library that supports over 40 unique chart types covering a wide range of statistical, financial, geographic, scientific, and 3-dimensional use-cases
• Plotly Python Open Source Graphing Library - Examples of how to make line plots, scatter plots, area charts, bar charts, error bars, box plots, histograms, heatmaps, subplots, multiple-axes, polar charts, and bubble charts
• Heatmaps with Plotly - How to make Heatmaps in Python with Plotly

Ipywidgets

• Ipywidgets documentation - ipywidgets, also known as jupyter-widgets or simply widgets, are interactive HTML widgets for Jupyter notebooks and the IPython kernel.
• Introduction to ipywidgets - in this tutorial video, learn about ipywidgets, a Python library for building interactive HTML widgets for your Jupyter browser.
• Ipywidgets Interact Function | ipywidgets Examples of Slider, Dropdown, Checkbox, Text Box - Video demo on how to make an ipywidgets slider, ipywidgets dropdown, ipywidgets checkbox, or an ipywidgets text box using Python code.

R

• Install R guide from Software Carpentries - R is a programming language that is especially powerful for data exploration, visualization, and statistical analysis. To interact with R, we use RStudio, which must also be installed separately from here
• Programming with R from Software Carpentries - this introduction to R is built around a common scientific task: data analysis
• R for Reproducible Data Analysis from Software Carpentries - write modular code and best practices for using R for data analysis
• R for Social Scientists - basic information about R syntax, the RStudio interface, how to import CSV files, the structure of data frames, how to deal with factors, how to add/remove rows and columns, how to calculate summary statistics from a data frame, and a brief introduction to plotting

MATLAB

• Using git with MATLAB - Introduction into using MATLAB and version control with git
• Programming with MATLAB - Lesson from the Software Carpentries on the basics of programming with MATLAB

Modular Code and testing

• Writing tests - lesson from CodeRefinery on automated testing
• Video testing lesson - recording from software testing workshop by Code Refinery
• Modular coding - modular code development from Code Refinery

Docker

• Installing Docker - installation instructions for Windows, macOS, and Linux
• Install WSL2 update - manual WSL2 update for Windows
• Dockerfile reference - information on how to write a Dockerfile

VSCode

• Setting up VSCode for Linux - guide to getting started using VSCode with Windows Subsystem for Linux

Continuous Integration

• GitHub Actions introduction course - an introductory course from GitHub on how to use GitHub Actions

Reproducible Research

• Reproducible Research material from Code Refinery - demonstrates how version control, workflows, containers, and package managers can be used to record reproducible environments and computational steps
• Reproducible Research video lesson from Code Refinery - Recorded video lesson from Code Refinery workshop in May 2021 on Reproducible Research material
• Data + Code + Software = PDF - Slides to an overview on how to integrate data and software into a PDF.