I cannot recommend enough the "John Lamping - The One Weird Trick for Analyzing Big Data ... Eyeball it Early and Often!" video:

https://www.youtube.com/watch?v=jYH8CQS6Ab0

One of the best tips, straight from a practitioner - from a former Google search ranking engineer who touched multiple other domains later in the career. Stop tuning knobs and watching metrics, look at the data!