Morningstar, known for its mutual-fund ratings, has hit a rough patch in its quest to become a big player in the bond-rating business.

In mid-November, bond investors got an unwelcome surprise from one of the main ratings firms in a hot corner of the bond market: About 25% of the bonds that it had rated were likely to be downgraded.

Several days later, after calls poured in from confused investors, ratings firm DBRS Morningstar Inc. backtracked and said it had made an error. It was instead likely to upgrade about 25% of the bonds and downgrade only about 3%.

“It...