Quantum randomness amplification protocols have increasingly attracted attention for their fantastic ability to amplify weak randomness to almost ideal randomness by utilizing quantum systems. Recently, a realistic noise-tolerant randomness amplification protocol using a finite number of untrusted devices was proposed. The protocol has the composable security against non-signalling eavesdroppers and could produce a single bit of randomness from weak randomness sources, which is certified by the violation of certain Bell inequalities. However, the protocol has a non-ignorable limitation on the min-entropy of independent sources. In this paper, we further develop the randomness amplification method and present a novel quantum randomness amplification protocol based on an explicit non-malleable two independent-source randomness extractor, which could remarkably reduce the above-mentioned specific limitation. Moreover, the composable security of our improved protocol is also proposed. Our results could significantly expand the application range for practical quantum randomness amplification, and provide a new insight on the practical design method for randomness extraction.