100 Machine Learning tips and TRICKs to celebrate

This channel is now partnered with Youtube! 🎉 Celebrate with me and these 100 machine learning Tipps! —————————————————————————————————————— Join my newsletter for more insights: 💌 —————————————————————————————————————— Missing Data ConvNext 2020 The Illustrated Transformer GANs Cycle GANs Numpy Einsum: Imagenet to Imagenet MissingNo Pandas Profiler Huber Loss: —————————————————————————————————————— 🎥 My Camera Gear 🎼 My Music —————————————————————————————————————— ⏱️ Timestamps 00:00 - Start 00:20 Learn about Shortcuts and Compression 00:31 Treat missing data correctly 00:50 Read the Convnext 2020 paper for CNNs 01:25 Let experts label your data 01:49 Learn about Transfomers 01:59 For regression don't forget R² 02:15 GANs are easier to train than you think 02:32 Get to know your data 02:44 Split out your test set asap 03:01 Transfer learning is great 03:23 Go with the basics 03:37 Tune your hyperparameters 03:48 Use Cross-validation & Baseline Models 04:17 Use Data Augmentation 04:31 Use Explainable AI 04:48 Be careful with benchmark results 05:02 Put your papers on Arxiv 05:15 Cut through the noise 05:25 Publish Your Code 05:49 Talk to Domain Scientists 06:16 Survey the Literature 06:38 Use benchmarks 06:50 Check for Class Imbalances 07:02 Build Trust through communication 07:19 Build Benchmarks for Credibility 07:38 Why class imbalance is difficult 08:04 Use Pytorch lightning 08:14 Never upgrade CUDA 08:22 Train your models online 08:36 Don't Overpromise Solutions 08:51 Overfit a small batch for debugging 09:03 Use Adam or SGD Optimizers 09:25 Set your gradients to None 09:37 Try Gradient clipping if you get NaNs 09:50 Fuse small operations 10:06 Reduce the batch size to replicate papers 10:16 Don't mix BatchNorm and biases 10:26 Pin Pytorch memory & Check your weight decay 10:45 Use gradient accumulation 11:06 Careful with Softmax 11:32 Use Mixed Precision 11:42 Inspect bad data points 11:52 Build redundancy in your MLOps 12:01 Pytorch async data loading 12:17 Use the Classification Report 12:28 Keras Lambda Layers 12:38 Don't use Random Forests for Feature Importances only 12:55 Use XGBoost and Neural Networks 13:05 Einsum is great! 13:25 Research Adjacent Fields 13:40 Hydra for Configs 13:54 MissingNo Library 14:04 Pandas Profiler 14:15 Paperswithcode 14:34 Try Unets 14:44 Use EarlyStopping 14:54 Set your Dropout right 15:04 Check out Profilers 15:14 Experience Replay 15:24 Use Schemas in production 15:34 Empty Pytorch and TF cache 15:44 Normalize your inputs 15:54 Use Robust Scalers 16:05 Find difficult to train samples 16:21 Arbitrary input sizes 16:34 Use GANs for real-world data 16:52 Set up Data pipelines 17:02 Use Confusion Matrixes & Find the maximum batch size 17:21 Use checkpoints on Colab 17:35 Learn the different model APIs 17:51 Debug with Tensorboard 18:01 Pre-allocate memory for dynamic tensors 18:13 Feature engineering 18:37 Random Forest can overvalue noisy features 18:48 Read the Docs 19:05 Ensemble models 19:15 Always think if a model should even be built 19:25 Remove correlated samples from training data 19:35 Dare move away from defaults 19:45 Log your experiments 20:02 Build smaller models 20:14 Change Kaggle Sorting 20:39 Learn from Kaggle 20:49 Make ablation studies 20:57 Check out regularization techniques 21:15 Learning Rate Scheduler 21:39 Don't overfit by hand 21:56 Create decorrelated validation and test sets 22:35 Create Tensors on device 22:45 Fix all randomness for publication 22:59 Visualize your training 23:18 Compare models with AIC 23:30 Publish your model weights 23:40 Look at your outputs 24:01 Huber loss 24:19 Trust domain scientists 24:50 Don't believe all old ML wisdom 25:04 Outro —————————————————————————————————————— 👋 Social 💙 Linkedin: 💜 Twitter: 🌍 Main Website: 🎁 Community: —————————————————————————————————————— 📝 Disclaimer Jesper Dramsch is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to and affiliated sites. Opinions my own. Not financial advice. Sponsors are acknowledged. For entertainment purposes only.