I very much appreciated comments left on Glassdoor by previous candidates when contemplating applying for the product manager position so I feel a duty to also contribute some of my own. Having read those slightly negative experiences I was of two minds whether I wanted to actually go through the process myself but since the role was a fairly good fit for my background I was pretty confident and decided to proceed. The first 3 interviews I had were all quite positive - the people I met were very capable individuals with an open/inclusive attitude so the interviews were quite interesting and enjoyable. This improved my impression of the company quite a bit. The last interview I had was a little different - a more formal, exclusive, “prove to me you have what it takes to be in this company” gatekeeper type experience - certainly not unusual by any means.
All interviews were focussed primarily on seeing if I knew what metrics to measure, how I might interpret results and what subsequent actions I might take based on those results. When in the last interview I mentioned that the sample size of an A/B test I had run was small enough so as to raise my concerns about the statistical significance of the results I was asked to provide the actual formula for calculating statistical relevance - I knew it was called p-value which I have used in the past when analysing results but being unable to describe the actual mathematical formula in the interview seemed to be interpreted as a bit of a black mark - at which point I started to wonder if I was in the wrong meeting. And that’s what the last interview was mostly about - looking for negatives to eliminate the candidate as opposed to earlier interviews which, whilst obviously validating basic pre-requisites, also looked to understand what else I could bring to the company.
Data analysis and data-driven business decision making aspects were heavily weighted in the interviews. I do not recall any questions that would elicit a better understanding of my job description aspects such as leadership, written communication skills, Japanese ability, start-up experience, web services technologies, product launch experience, competitor/market research - all of which I think I have a pretty solid track record on. For a company so devoted to data-driven decisions it seemed to me somewhat ironic that what was being measured didn’t really align well with the actual requirements.
One would think that having conceived of, led, launched multiple products and built, growth-hacked my own B2C web startup in Japan this would lend some credibility to my ability to analyse and drive success but the feedback we received was that my data analysis experience was insufficient. Frankly, this is just nuts. It should have been obvious that I would have been deadly on the particular “Indeed way” of data analysis within a week of joining the company - the learning curve for such being far far less than other critical product manager skills like leadership, innovation, and Making Stuff Happen.
My advice for potential candidates would be ignore most of the job description. Even if you have an exceptional product management track record, unless you have substantial experience designing and interpreting experiments on a *large* website I think there is a high probability you will be eliminated late in the interview cycle after you have already invested a lot of time and energy. Had I known this I would not have applied. Naturally both my recruiter and myself were very surprised with the outcome.