Side-channel resistance is nowadays widely accepted as a crucial factor in deciding the security assurance level of cryptographic implementations.In most cases,non-linear components (e.g.S-Boxes) of cryptographic algorithms will be chosen as primary targets of side-channel attacks (SCAs).In order to measure side-channel resistance of S-Boxes,three theoretical metrics are proposed and they are reVisited transparency order (VTO),confusion coefficients vari-ance (CCV),and minimum confusion coefficient (MCC),respectively.However,the practical effectiveness of these metrics remains still unclear.Taking the 4-bit and 8-bit S-Boxes used in NIST Lightweight Cryptography candidates as concrete examples,this paper takes a comprehensive study of the applicability of these metrics.First of all,we empirically investigate the relations among three metrics for targeted S-boxes,and find that CCV is almost linearly cor-related with VTO,while MCC is inconsistent with the other two.Furthermore,in order to verify which metric is more effective in which scenarios,we perform simulated and practical experiments on nine 4-bit S-Boxes under the non-profiled attacks and profiled attacks,respectively.The experiments show that for quantifying side-channel resistance of S-Boxes under non-profiled attacks,VTO and CCV are more reliable while MCC fails.We also obtain an interesting observation that none of these three metrics is suitable for measuring the resistance of S-Boxes against profiled SCAs.Finally,we try to verify whether these metrics can be applied to compare the resistance of S-Boxes with different sizes.Unfortunately,all of them are invalid in this scenario.