A question of power?

nuigal

Hi there,

Just wondering if anyone can help explain this problem. In short I have used a stepwise regression with A LOT of variables and a small sample size.(I realise this is less than ideal but I'm not in a position to change either) Despite this I have found significance and the R2 (.567) value seems to provide good power (.4) ...based on an online power calculator.

Is there a reason why this might have happened. As in...are the sample size and number of predictors able to influence this to such an extent or is the test showing actual significance.

Any advice would be very welcome!

Thank you!

Black Swan

Sample size?

Ostrom

The inclusion of additional predictors will inflate the r^2 automatically so I wouldn't rely solely on that as an indicator. Is this a time series regression? You can get inflated t-stats and model diagnostics quite easily if so. How are the coefficient t statistics? What about the model F?

I'm deeply sceptical about stepwise methods - a little of my own bias, but you're abdicating an awful lot of the model specification to the computer. Check for colinearity also - the mean VIF will give you a quick indication.

nuigal

Hi there sample size was 50 and it wasn't a time series regression. I think I've managed to sort out the problem since I last posted. My power is pretty awful but I'm going to have to run with it anyways! Thank you both for posting in the meantime! Much appreciated and sorry for not replying sooner!

nuigal

Hi there. I ran 3 regression and the adjusted r2 varied from .23, .28 and .40. With .40 being the one I was most interested in. Because of what I was asking I had to include ~20 variables which was far less than ideal with my sample size! I cannot reduce this so I have to report what I got. Post hoc power is < .2.

nuigal

Very little theoretical background so the investigation is exploratory. Hierarchical wouldn't be appropriate.

nuigal

Might try that. Thanks for the advice!