Many talk about the value of research and the gems it can help uncover. The topics for learning and potential discoveries are vast and are often backed by countless publications and tutorials available online. Yet, it seems that not enough attention is given to the “how” of the process itself. Research can take many forms, have different objectives and timespans and at the end of the day may get discouraging.
Disclaimer: the shared links will have a strong flavor of AI, but some things should apply to other fields as well.
For great general advice read Lessons from My First Two Years of AI Research written by Tom Silver, PhD student at MIT, 2018.
Here are my condensed highlights from it:
“A bad visualization will require recalling in detail the code you wrote to produce it; a good one will scream an obvious conclusion.”
“Backtracking is forward progress.”
For more valuable insights and structure read the original article.
Now that you hopefully feel even more excited to dive into reading, watch the video tutorial by Yannic Kilcher, PhD Student at ETH Zurich, How I Read a Paper: Facebook’s DETR. He has a lot of well-curated reviews of deep learning research, but I feel like this one is worth watching first. In this video he shares his approach and shows which sections he looks at first and which he usually skips, which require thinking critically or going over twice, etc.
Small tip: Many researchers claim that a “Related Work” section is irrelevant, but I feel like sometimes it’s actually a great way to start unraveling the topic, since you can follow through the references with a bonus: authors’ assessment of them. Alternatively, look for a good survey paper.
The tool used in the video is OneNote.
A few days into reading and papers begin to pile up. Even if you take notes, unless you organize them properly they get lost and forgotten. It’s hard to remember what was unique about each publication. Also, you might want to highlight some text inside of a paper itself. Save a tree if you can and use some software instead. There are benefits that come with it, such as an opportunity to organize your ideas into a hierarchical structure, to attach notes, collaborate with a group of people, etc.
Let’s go over a few options out there.
These are just a very few. Invest some time into finding the one that works for you, it will definitely pay off!
Check out how other people solve similar problems with code, then build your own thing in a fork or in a new repository. Just don’t forget to check the license and to cite the authors if you are using someone else’s work.
Papers With Code is a great place to do that. They also recently partnered with arXiv where now you’ll find a Code tab for each publication, so see if authors left a link to Official Code.
Visit this page regularly to keep yourself up-to-date and to check for new methods organized by areas of Machine Learning.
This page may introduce you to 3 very valuable notions:
You can also search for relative leaderboards on Kaggle. But be extra mindful about leaderboards in general. As Chip Huyen writes in her booklet Machine Learning Systems Design in the Performance requirements section:
“A few percentage points might be a big deal on a leaderboard, but might not even be noticeable for users.”
And also
“…a model can do better than the rest just by chance (AI competitions don’t produce useful models, Luke Oakden-Rayner, 2019).”
To conclude I’d like to wish you good luck with your project. You are going to be inspired, excited, daunted, frustrated and all over again. It’s all part of the process. Just remember: discovering new things is fun! 🎉