Testing Profession Skill-Ups

When I was preparing to work on the Realm First: Illustrious Cooking achievement I found a formula to work out the odds of getting a skill-up depending on when a recipe turned yellow, when it turned grey, and current skill level. I didn't find any proof of this formula anywhere or anyone really giving any justification for it. I found it in a few spots and ran with it, but I needed far less than I thought I would to max cooking. Sky needed far fewer metagems than he thought he would to max jewelcrafting, too. Did we just get a little lucky or is the formula wrong?

I want to find out by testing the formula to see if it stands up for at least one recipe. The issue then becomes that I need a lot of repetition at the same skill level to have any confidence in estimating the true odds of a skill-up at that skill level. My first thought was to just unlearn cooking over and over since it has a recipe at skill 1 that uses only vendor materials. Unfortunately it turns out you can't unlearn secondary professions, so that won't work. I could delete the character every time but you need to be level 5 to learn cooking so I'd need to level the new character every time in addition to having to transfer funds to buy the cheap vendor mats. Alternatively I could run tailoring off linen cloth or engineering off rough stone if I could gather up enough stuff. Tailoring has the advantage that I could make actual greens to disenchant to mitigate some of the cost.

How much stuff is enough stuff? The answer to that question depends on how confident I want to be. If I was testing a single event (like a die or a coin) I'd probably want to be sure within half a percent 99% of the time. Now, n = Z^2/(4E^2) where Z is pulled from a table of values for normal distributions and E is the maximum error I'd accept so I'd need to flip that coin n = (2.5759)^2/(4*(.005)^2) = 66353. That's a very big number. There's an added wrinkle for me as well in that I'm not testing a single event, I'm testing like 60 of them. There's a different probability at every skill level, so I'm estimating the odds of 60 numbers, not 1 number. To maintain that level of confidence would require 133k linen cloth per skill level. Even worse, the first few yellow points will give me a skill-up right away so getting 66k trials on them requires powering the first 24 orange ticks almost that many times as well.

What I can do, though, is relax my confidence requirements. Bolt of Linen Cloth has 25 skills between yellow and grey, so I'm expecting a 4% drop between each skill level. So if I set my maximum error to 2% I should still see the gradation between different skill levels. With 25 numbers here I should see if the formula is way wrong or not as long as most of my estimates are good, so I don't need the 99% confidence either. 95.45% (two standard deviations) seems like it should be more than good enough. With those new numbers n = 2500. Ugh. Nevermind the fact I cant force the same number of trials for each skill level, plowing 5k linen into each of the 25 skill points is absurd. We're talking a third of the linen needed to open the gates of Ahn'Qiraj just to run this test. That's not going to happen.

What if I assume I want the last skill point to have the 2500 iterations? Then the first one only gets 100 iterations. That really hurts my confidence in the first few goes but does bring my needed linen down to 70k. Is this something I can do or should I just give up? I do also get to make something from the bolts, so I get to run half again as many trials on Brown Linen Pants. 70k seems semi-reasonable but not something I'm going to farm myself. I think I'll just start buying cheap linen on the AH for a while and see how many I end up with.

I can't start now anyway, as I need a way to collect the data. The easiest way to do this has to be writing a LUA mod to output information each time something is crafted. Does the language have access to the right information? Only one way to find out, and that's to actually learn to write a WoW mod.