The use of cellulosic reinforcement fillers, including cellulose and holocellulose, in the development of sustainable biopolymer composites has become increasingly essential and continues to attract significant attention in the composite industry. This study aimed to improve the structural and morphological characteristics of the polyhydroxybutyrate (PHB) matrix by incorporating cellulosic fillers-namely, alpha-cellulose and holocellulose produced via a green processing method-and to evaluate the effect of hemicellulose, present in holocellulose and exhibiting compatibilizing capability, on the overall performance of PHB-based blends. For this, the PHB matrix was first dissolved in chloroform, after which the cellulosic fillers were incorporated into the PHB-chloroform mixtures at 1 wt.% to provide the best homogeneous fiber dispersion. The PHB and cellulosic filler mixtures were blended at 500 rpm with a magnetic mixer for 30 min, and the resulting composite was cast onto a Teflon plate. Scanning electron microscopy (SEM), X-ray diffraction (XRD), and Fourier transform infrared (FTIR) spectroscopy were used to characterize the morphological and structural analysis of the obtained biopolymer-based composites. Thermogravimetric analysis (TG-DTG) was used to determine the thermal properties. The results obtained confirmed the presence of cellulosic fillers in the PHB matrix using FTIR, XRD, and SEM. In contrast to holocellulose, alpha-cellulose in the PHB matrix was shown to create a more organized structure. Both alpha-cellulose and holocellulose reinforcements were found to have similar effects on the thermal properties of the PHB matrix. Compared with neat PHB, the amount of residual char was found to be more than 36-fold in the sample containing alpha-cellulose and more than 41-fold in the sample containing holocellulose.