To security support large-scale intelligent applications, distributed machine learning based on blockchain is an intuitive solution scheme. However, the distributed ma-chine learning is difficult to train due to that the corresponding optimization solver algo-rithms converge slowly, which highly demand on computing and memory resources. To over-come the challenges, we propose a distributed computing framework for L-BFGS optimi-zation algorithm based on variance reduction method, which is a lightweight, few additional cost and parallelized scheme for the model training process. To validate the claims, we have conducted several experiments on mul-tiple classical datasets. Results show that our proposed computing framework can steadily accelerate the training process of solver in ei-ther local mode or distributed mode.