Williams, making an appearance at the year-end championship for the first time since 2009, got a break back in the fifth game, but Pliskova broke twice more to take the opener.
Williams had an opportunity to get a foothold in the match in the second set, but Pliskova fought off break points in the third game.
2 seed Muguruza broke five times, while French Open winner Ostapenko, seeded seventh and making her debut at the event, capitalized on three out of eight opportunities.
