Power Struggle

I got an email from Kevin Cummins (UCSD Medical School), who was playing with my powerlog program. powerlog is supposed to compute the necessary sample size to achieve a given power in logistic regression with a continuous predictor. You tell the program the proportion of one's at the mean of a continuous predictor (p1) and the proportion of one's at the mean plus one standard deviation (p2). I basically translated a SAS macro program by Michael Friendly (York University) who, in turn, referenced two formulas from Agresti's, An Introduction to Categorical Data Analysis (pg 131).

Kevin's email pointed out that my program produced some strange sample sizes. For some values of p1 and p2 the program was generating negative sample sizes and there were also times when sample size increased when the effect size increased. This, of course, is not a behavior you would expect.

The problem with the negative sample sizes was easy to correct once I spotted the error. There was a calculation that involved exp(-`lambda'^2) (the funny single quotes are Stata's way of indicating a macro variable). The way it supposed to work was that lambda was squared, then made negative and then exponentiated. The way I wrote it, lambda was first made negative, then squared and finally expopnentiated. An extra set of parentheses solved the problem.

The second problem was much more difficult to diagnose. I verified that the SAS macro program had the same problem of increasing sample size with increasing effect size under certain conditions. The algorithm used was from Hsieh (1989). So I did what I should have done in the first place, get a copy of the article from the library. I verified that Agresti, the SAS macro program and my Stata ado implemented the algorithm as it was written. Then, I did some online searching and found a much later Hsieh et al (1998) article which compared several different methods for estimating sample size for logistic regression. In the conclusion section of the article there was the statement that the algorithm should not be used for odds ratios greater than 3 or less than 1/3. With large odds ratios you can get some unbelievable sample size estimates. For example, with an odds ratio of 81 the program produces a suggested sample size of 1.7e+11.

So, right now I'm trying to decide whether to just restrict the program to odds ratios between 1/3 and 3 or to just sit on the program for a bit and search for a better algorithm that I can implement.

There will be a lot more items on power analysis coming up in the near future because we will be putting on a series of presentations on this topic in the Spring. We have an outside speaker (non-UCLA), Jason Cole (Consulting Measurement Group), who volunteered to do presentations on intermediate and advanced issues in power analysis. We will kick things off with an introduction to power analysis by our own Christine Wells. We will announce the dates and times on our ATS Stat web page.

pbe