A Data Science Approach in Product Analytics

  • siddharthray123

    SSC Veteran

    Points: 217

    Comments posted to this topic are about the item A Data Science Approach in Product Analytics

  • amd.repetto

    Ten Centuries

    Points: 1033

    Great article, points extra for mentioning Uruguay 🙂

    A. Mauricio Repetto
    ML Engineer

  • ktflash

    Ten Centuries

    Points: 1215

    Damn i should get into R

    I want to be the very best
    Like no one ever was

  • Jeff Moden

    SSC Guru

    Points: 997311

    I think that one of the most important parts of all of this can be found in the conclusion section of the article.  In particular, I'm talking about the following paragraph....

    It is common to see non-significant test results after conducting an A/B test. What constitutes significance comes from product teams who take the 'cost of making a change' into consideration. Since a lot of the times this cost can be large, even if the test is barely a success, the product team may decide to go against making that change.

    It definitely IS common to seen "non-significant test results" and that's actually a HUGE problem because of all the time already spent on developing a change and then going through A/B and a whole bunch of other testing.

    What is that HUGE problem?  It can be any one of any combination of the following...

    1. Poor evaluations of perceived problems.  No change was actually needed.
    2. Poor code or poor implementation of code.  People just couldn't get out of the same rut others were in when it came to better code and how it should be implemented.
    3. Really short schedule.  No one actually had time to make good changes and so they did as little as possible.
    4. Really short schedule.  People didn't actually have the time to understand the original problem well enough to improve on it.
    5. Mistake... someone forgot to do a code regression check or even forgot to check-in code.
    6. Not likely but maybe the original code was the optimal code (or screen, whatever) to begin with and someone decided to make a change because they didn't have that understanding or their decision making was based on insufficient data.
    7. Someone thought it was their job to "drive change".  One of my favorite laughs comes from a quote my cousin repeated from his old place of work.  Apparently some hungry manger said "We have to do something even if it's wrong". <Headdesk>

     

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
    "Change is inevitable... change for the better is not".
    "Dear Lord... I'm a DBA so please give me patience because, if you give me strength, I'm going to need bail money too!"

    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • siddharthray123

    SSC Veteran

    Points: 217

    Glad you liked it! Thank you!

  • siddharthray123

    SSC Veteran

    Points: 217

    Great points there Jeff! Thanks for reading.

Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply