The composite films of GaSb nanoparticles embedded in SiO2 matrices were fabricated by radio-frequency magnetron co-sputtering. Transmission electron microscope and X-ray diffraction pattern indicate that the GaSb nanoparticles were uniformly dispersed in SiO2 matrices. Room temperature transmission spectra exhibit a blue shift of about 2.73 eV. The blue shift increases with decreasing size of GaSb nanoparticles, suggesting the existence of quantum size effects. Room temperature Raman spectra show that there is a larger Raman peak red shift and broadening of the composite films than that of bulk GaSb. This phenomenon is explained by photon confinement effect and tensile stress effect.