ETL With Apache Airflow
Outline of the Project
Posts by Year
2022
2021
Color Theory: Triadic
I’ve recently added somemissing .SCSS files tomy websites github repository, which will now let write my own theme. I’ve taken inspiration from flower arrang...
On the topic of your brain and how you ruined it
Since graduating high school six years ago, I’ve always felt like my long-term memory and ability to meaningfully learn new concepts has been declining. Anec...
Week of August 9th
What I did: Monday: Kaggle work with giant spotify dataset Tuesday: Added use of the Spotify API, spotipy to the notebook. Weds - Sat: Added recommend...
Let’s Talk Transfer Learning
One-shot transfer learning with regression
Week of Aug 2nd
I created a really nice template for ML projects that I hope BNL will continue to use for a long time.
Week of July 1st
What I did Probability Problems
Week of June 14th
What I did Monday: Fixed MSD Values of particle averaged structure. I was double counting some bonds before. Tuesday: Neural Network now predicts 5 feat...
Week of June 6th
What I did Sunday: Trained the neural network on the skew-norm dataset and spent some time playing around with the architecture and hyperparameters. Mon...
Week of May 24th
Examples of using pandas .pipe()
Week of May 24th
Hurst Oxponent, Time Series, etc.
Week of May 17th
What I did Monday: Energy mesh interpolation codes Tuesday - Thursday: Felt dead from the seconds shot of the Moderna covid vaccine. Didn’t do much. I’v...
Week of May 10th
What I did Monday: Finished script to generate thousands of real-disordered structures. Tuesday: Wrote a script to loop through the results from the sim...
Week of May 2nd
What I did Monday: NN data-loading Tuesday: NN pre-processing Wednesday: NN pause. I need to test that my gaussian averaging method is valid enough to...
Week of April 25th
What I did Sunday: Thesis chapter \(\LaTeX\) writing Monday: Thesis chapter \(\LaTeX\) writing Tuesday: More writing, but I also redid some figures co...
Week of April 11th
Skewnorms and comparing simulations to experiment data
Week of April 4th
Website building, mathematica rendering, and lots of seaborn manipulation
Week of March 29th
What I did Monday: Got FEFF to work. Read paper: “Probing Atomic Distributions in Mono- and Bimetallic Nanoparticles by Supervised Machine Learning” Nano ...
Week of March 22nd
What I did Monday: Train ride back to Boston. Wrote python script to generate .inp files with different radially dispalced atoms Tuesday: Cleaned up the...
Week of March 15th
What I did Monday: Worked on jupyter notebook. See plotly plot from last week Tuesday: Feff9 on linux debugging all day conference call. Wednesday: Pr...
Week of March 8th
What I did Sunday: XAFS Thesis writing Monday: Traveling Tuesday: FEFF Codes and disorder in XANES paper reading Wednesday: 2010 Phys Review B, “Eff...
Week of March 1st
This blog will now partly shift from what I learned as a data scientist to what I’ve learned as a scientist, to keep track of my thesis progress. Of course s...
Paygap
<!– — title: “On the Importance of Data Literacy in Decision Making” date: ‘2021-02-10’ categories: miscellaneous tags: statistics
Week of January 10th
I’m getting close to my final exams so most of my time will go towards studying. Accordingly, expect a hiatus from blog postings.
2020
Week of December 27th
Bayesian analysis of Amazon reviews in R and the most important functions in Pandas
Week of December 13th
GRU’s, strings, and batch normalization. I didn’t post anything last week because I was busy with grad school work and traveling – I flew back home to New Yo...
Week of November 29th
Homo and heteroskedasticity. When to use NumPy vs. Pandas.
Week of November 22nd
R equivalencies in Python.
Entropy, Information, and Guess Who
Decision trees are one of the first data structures you learn in computer science (graphically, in practice you might learn them as a linked list), but you m...
Week of November 8th
Probability and statistics riddles
Week of November 1st
Stats riddles, Covariance Matrix, etc.
Week of October 25th
SVD and Eigendecomposition, XGBoost, etc.
Week of October 18th
SQL, Naive Bayes, Python tips
Week of October 11th
SQL, precision, recall, and fun with Numpy
Interpreting Conflicting Statistics from the Same Data
Original Paper
Weeks of October 6th
Note, last week is missing. Most of what I did is included in the prior week, and the rest is included here. It wasn’t enough to merit its own week, because ...
Weeks of September 20th
Sensitivity, CSV reading, and Pandas
Week of September 13th
KRR, SVD, and PCA
Weeks of August 30th and September 6th
I’ve decided to combine my weekly’s for these two weeks because I’ve had to spend a lot of time traveling (Germany->Strasbourg->Paris->Montpellier) ...
Week of August 16th
What I did
Week of August 9th
What I did
Activation Functions Review
Sigmoid
Week of August 2nd
What I did
Week of July 26th
What I did
Week of July 12th
What I did
Week of July 5th
What I did
Week of June 28th
What I did
Week of June 21st
What I did
Week of June 14th
What I did
Week of June 7th
What I did
PyTorch, A Quick Intro
Pytorch uses tensors instead, similar to a numpy array
Week of May 30th
What I did
Taylor Expansion of the RMS Prop Update Algorithm
RMSprop is similar to gradient descent with momentum, but takes into consideration the sign of the gradient like this:
The Dance of Forward Pass and Backprop
In order to implement a neural network, a solid understanding of Linear Algebra is needed for the forward pass, whereas vector calculus is needed for backpro...
Week of May 24th
What I did
Week of May 17
What I did
WSL2: VM’s in Windows are now Obsolete
In my previous miscellaneous post, I wrote about why I gave up on WSL1 and switched to running ubuntu in a virtual machine. After looking into WSL2 more, I c...
To VM or Dual Boot
Update: Using a virtual machine has not been as smooth as I expected. I’ve ended up with some weird problems after a few days. For example: sometimes my trac...
Week of May 3rd
What I did
First Post: Format of this blog
This is my first post, and I would like to explain my plan for this blog. During my summer work with my undergraduate thesis advisor, I would have a weekly m...