The purpose of this study was to achieve a sustained release of hydrophilic L-ascorbic acid 2-phosphate magnesium (ASP) from electrospun polycaprolactone (PCL) scaffolds, so as to promote the osteogenic differentiation of stem cells for bone tissue engineering (TE). ASP was loaded and electrospun together with PCL via three electrospinning techniques, i.e., coaxial, emulsion, and blend electrospinning. For blend electrospinning, binary solvent systems of dichloromethane-methanol (DCM-MeOH) and dichloromethane-dimethylformamide (DCM-DMF) were used to achieve the desired ASP release through the effect of solvent polarity and volatility. The scaffold prepared via a blend electrospinning technique with a binary solvent system of DCM-MeOH at a 7: 3 ratio demonstrated a desirable, sustained ASP release profile for as long as two weeks, with minimal burst release. However, an undesirable burst release (similar to 100%) was observed within the first 24 h for scaffolds prepared by coaxial electrospinning. Scaffolds prepared by emulsion electrospinning displayed poorer mechanical properties. Sustained releasing blend electrospun scaffold could be a good potential candidate as an ASP-eluting scaffold for bone tissue engineering.