After the release of Throwing Muses' fifth album, Red Heaven, Kristin Hersh, the band's lead singer/songwriter, took a break from the group and issued her first solo album, the acoustic Hips and Makers, in early 1994. Thanks to the airplay the single "Your Ghost" -- a duet with R.E.M.'s Michael Stipe -- received, the album sold more copies than any of the Muses' releases. Later that year, Hersh also released the Strings EP, which featured versions of selected songs from the album recorded with a string quartet, and did a solo tour. Despite her success as a solo artist, she kept the Muses going as well; they released their next record, University, in February 1995. Throwing Muses and Hersh as a solo artist moved from Sire to Rykodisc, forming the boutique label Throwing Music in 1996; the label's first release was the final Muses album, Limbo, which appeared that summer. Before the Muses hit the road in support of Limbo, Hersh began work on her second solo album, wrapping it up in early 1997.

At the completion of the Limbo tour, Hersh disbanded Throwing Muses, claiming that it was no longer economically feasible to continue with the band. The acoustic-based Strange Angels, her first post-Muses album, was released by Rykodisc in February 1998. That year, she also released Murder, Misery and Then Goodnight, a collection of lullabies and Appalachian folk songs, as a Throwing Music exclusive. In 1999, Hersh and Throwing Music returned to her original label, 4AD, for Sky Motel, 2001's Sunny Border Blue, and 2003's The Grotto. Hersh also established the Works in Progress series, a subscription service of exclusive rarities, through Throwing Music's website.

Hersh released another Throwing Muses album in 2003, then collaborated with Muses bassist Bernard Georges in 50 Foot Wave, a more rock-oriented expression of her sound, for 2004's self-titled EP, 2005's Golden Ocean, and 2006's digitally released Free Music EP. In 2007, Hersh released another solo album, the intense Learn to Sing Like a Star. Cats and Mice, an unedited performance recorded at Yoshi's in San Francisco, was issued in 2010. That same year, Hersh published her memoir, Rat Girl, which focused on events in her teens from 1985 to 1986. Also in 2010, she released the album Crooked via her website. 2013 saw the release of a new Throwing Muses LP, Purgatory/Paradise, and in 2015 Hersh published a new semi-memoir titled Don’t Suck, Don’t Die: Giving Up Vic Chesnutt, which focused on her long friendship with the late singer/songwriter. Wyatt at the Coyote Palace, Hersh's first new collection of solo recordings in six years, dropped in November 2016. Two years later she returned with her 11th solo record, and first for British independent label Fire Records. Possible Dust Clouds featured ten tracks with "LAX" and "No Shade in the Shadow" appearing in advance of the album's release.