I’ve been there, we wanted to release a feature, it kept coming back with issues that made it perform much worse than control, after 5 or so iterations with bug fixes it came back positive.
It took a lot of analysis and time to clarify to higher ups that we weren’t just P-hacking , but at least they were concerned about that.
It took a lot of analysis and time to clarify to higher ups that we weren’t just P-hacking , but at least they were concerned about that.