Definition of pragmatic (Merriam-Webster)
1: relating to matters of fact or practical affairs often to the exclusion of intellectual or artistic matters: practical as opposed to idealistic.
But can you really exclude all matters of the heart? Can you remove all politics from technical decisions? Maybe thinking so is being idealistic! Being pragmatic means promoting fact based information and practical aspects over other considerations, but other considerations can also be facts to deal with. Being truly pragmatic might not be so much cutting the link between the mind and the heart but more finding the right balance where one privilegiate facts while still accounting for other pursuits such that in the end we get a practical solution.
This week’s mentoring and post is based on the eight and last chapter of “The Pragmatic Programmer: From Journeyman to Master”: Pragmatic Projects. This chapter wraps up everything we learned up to this point into a nice view and provides a good summary of the learnings.
If you want to live the pragmatic data scientist (or programmer) life, you’ve got to implement pragmatism in all aspects of your team workings. You ought to automate every repetitive aspect and make sure testing of your artifacts are automated as well. No manual steps that need to be repeated over time.
Pragmatic teams should not allow for Broken Windows (see the first post of the series). Second, you should always monitor your “environment” and not let it degrade over time. Even a slow degradation if left unchecked will amount for large problems in the future. This can creep up in many ways: data growing linearly may be a challenge if left unnoticed. You may need to adjust your resources as time passes, why not automate that? If you decide on that path, make sure you put some limits on the increase factor allowed over time if you don’t want to end up with a huge bill! Now what about exponential growth of data? If that growth follows your “customer” base, that might not be a financial problem to adjust the resources, but what if it’s not? You should address the issue as soon as possible. Maybe there are other ways to aggregate the data and for it to still be relevant.
As we have seen in previous chapters, communication is king. Communication within the team as well as communication with stakeholders and other parts of the organization. I often found that the best advice this book provides on that subject is to create a brand. When you start a project, give it a name and always refer to it. If it can by itself create a mental image of what you try to achieve the better.
Then there are a bunch of reminders. Don’t repeat yourself! Refactor to eliminate those repetitions. Create orthogonality, not by organizing around job functions, but around functionality.
Value testing, and repeatable testing. Value as well a good data scientist level documentation. Not documentation made for the sake of a process, or for other parties that won’t read it (well if they are to read it, do it, but make sure it answers their needs, no other useless extra). In a few months when/if you need to get back to that piece of code or algorithm you design, will you be able to remember all the small decisions you made? I bet not. And if you leave the organization, will the team still be able to maintain and enhance that functionality? Have documentation as close to the code as possible. If you have concepts that spans a lot of code e.g. system level information, make it meaningful for yourself and your fellow data scientists firsts. Make it accessible, and don’t repeat information found at levels closer to the code or algorithm. I especially like the tip 67: “Treat english as just another programming language”. This means that all that applies to your code should apply to your documentation…
And most importantly, remember that good enough is good enough. There is no perfection in this world and even if you could achieve it, the goal would move away the next day.
This concludes our series on reading through “The Pragmatic Programmer: From Journeyman to Master”. I hope this can help you become a better programmer for data science! Keep on programming, keep on practicing!