Collagen fibrils are the main structural element of connective tissues. In many tissues, these fibrils contain two fibrillar collagens (types I and V) in a ratio that changes during tissue development, regeneration, and various diseases. Here we investigate the influence of collagen composition on the structure and rheology of networks of purified collagen I and V, combining fluorescence and atomic force microscopy, turbidimetry, and rheometry. We demonstrate that the network stiffness strongly decreases with increasing collagen V content, even though the network structure does not substantially change. We compare the theological data with theoretical models for rigid polymers and find that the elasticity is dominated by nonaffine deformations. There is no analytical theory describing this regime, hampering a quantitative interpretation of the influence of collagen V. Our findings are relevant for understanding molecular origins of tissue biomechanics and for guiding rational design of collagenous biomaterials for biomedical applications.