Abstract
Methods of incorporating a ridge type of regularization into partial redundancy analysis (PRA), constrained redundancy analysis (CRA), and partial and constrained redundancy analysis (PCRA) were discussed. The usefulness of ridge estimation in reducing mean square error (MSE) has been recognized in multiple regression analysis for some time, especially when predictor variables are nearly collinear, and the ordinary least squares estimator is poorly determined. The ridge estimation method was extended to PRA, CRA, and PCRA, where the reduced rank ridge estimates of regression coefficients were obtained by minimizing the ridge least squares criterion. It was shown that in all cases they could be obtained in closed form for a fixed value of ridge parameter. An optimal value of the ridge parameter is found by G-fold cross validation. Illustrative examples were given to demonstrate the usefulness of the method in practical data analysis situations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, S.E., Coffey, B.S., & Byerly, R. (2002). Formal organizational initiatives and informal workplace practices: Links to work-family conflict and job-related outcomes. Journal of Management, 28, 787–810.
Anderson, T.W. (1951). Estimating linear restrictions on regression coefficients for multivariate normal distributions. Annals of Statistics, 22, 327–351.
Breiman, L., & Friedman, J.H. (1997). Predicting multivariate responses in multiple linear regression. Journal of the Royal Statistical Society, Series B, 59, 3–54.
Chinchilli, V.M., & Elswick, R.K. (1985). A mixture of the MANOVA and GMANOVA models. Communications in Statistics, Theory and Methods, 14, 3075–3089.
D’Ambra, L., & Lauro, N. (1989). Non symmetrical analysis of three-way contingency tables. In R. Coppi & S. Bolasco (Eds.), Multiway data analysis. Amsterdam: North-Holland.
Duxbury, L., Higgins, C., & Lee, C. (1994). Work-family conflict: A comparison by gender, family type, and perceived control. Journal of Family Issues, 25, 449–466.
Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap. Boca Raton: CRC Press.
Hoerl, K.E., & Kennard, R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.
Groß, J. (2003). Linear regression. Berlin: Springer.
Lambert, Z.V., Wildt, A.R., & Durand, R.M. (1988). Redundancy analysis: An alternative to canonical correlation and multivariate multiple regression in exploring interset associations. Psychological Bulletin, 104, 282–289.
Legendre, P., & Legendre, L. (1998). Numerical ecology, 2nd English edn. Oxford: Elsevier.
Reinsel, G.C., & Velu, R.P. (1998). Multivariate reduced-rank regression: Theory and applications. New York: Springer.
Sparrow, P.R. (1996). Transitions in the psychological contract: Some evidence from the banking sector. Human Resource Management Journal, 6, 75–92.
Takane, Y., & Hwang, H. (2002). Generalized constrained canonical correlation analysis. Multivariate Behavioral Research, 37, 163–195.
Takane, Y., & Hwang, H. (2006). Regularized multiple correspondence analysis. In J. Blasius & M.J. Greenacre (Eds.), Multiple correspondence analysis and related methods (pp. 259–279). London: Chapman and Hall.
Takane, Y., & Hwang, H. (2007). Regularized linear and kernel redundancy analysis. Computational Statistics and Data Analysis, 52, 394–405.
Takane, Y., & Shibayama, T. (1991). Principal component analysis with external information on both subjects and variables. Psychometrika, 56, 97–120.
Takane, Y., & Yanai, H. (2008). On ridge operators. Linear Algebra and Its Applications, 428, 1778–1790.
ten Berge, J.M.F. (1993). Least squares optimization in multivariate analysis. Leiden: DSWO Press.
ter Braak, C.J.F., & Šmilauer, P. (1998). CANOCO reference manual and user’s guide to Canoco for Windows. Ithaka: Microcomputer Power.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.
Van den Wollenberg, A.L. (1977). Redundancy analysis: Alternative for canonical analysis. Psychometrika, 42, 207–219.
Velu, R.P. (1991). Reduced rank models with two sets of regressors. Applied Statistics, 40, 159–170.
Yuan, M., Ekici, A., Lu, Z., & Monteiro, R. (2007). Dimension reduction and coefficient estimation in multivariate linear regression. Journal of the Royal Statistical Society, Series B, 69, 329–346.
Author information
Authors and Affiliations
Corresponding author
Additional information
We thank Jim Ramsay for his insightful comments on an earlier draft of this paper. The work reported in this paper is supported by Grants 10630 from the Natural Sciences and Engineering Research Council of Canada to the first author.
Rights and permissions
About this article
Cite this article
Takane, Y., Jung, S. Regularized Partial and/or Constrained Redundancy Analysis. Psychometrika 73, 671–690 (2008). https://doi.org/10.1007/s11336-008-9067-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-008-9067-y