The Baltimore Orioles selected Olson in the first round, with the fourth pick, of the 1988 MLB draft, and he signed for a $200,000 signing bonus[4] before making his major league debut on September 2, 1988. A reliever, he threw what baseball historian Sheldon Stewart referred to as a "blazing fastball and devastating curve".[5]
Selected to the All-Star team in 1990, Olson set a club record of 37 saves during the season and collected 31 and 36 in the next two years. On July 13, 1991, Olson combined with 3 other Baltimore pitchers in a no-hitter against the Oakland Athletics.[6] In August 1993, Olson suffered a torn elbow ligament injury that sidelined him for the rest of the year. He finished with 29 saves and a career low 1.60 ERA, but Baltimore opted not to take a risk with him and signed Lee Smith as their new closer. Olson struggled with a succession of injuries over the next years, playing for seven different teams from 1994 to 1997.
In 1998, Olson enjoyed a fruitful comeback with the expansion Arizona Diamondbacks. He set a franchise record of 30 saves (broken by Byung-hyun Kim in 2002) and was also part of a rare feat. On May 28, with Arizona leading the San Francisco Giants 8–5, Olson began the bottom of the ninth inning by striking out Darryl Hamilton, but the Giants then loaded the bases with two walks and a hit before Stan Javier had an RBIgrounder that made it 8–6. After pinch-hitterJ. T. Snow walked to load the bases, managerBuck Showalter ordered Olson to intentionally walk Barry Bonds, forcing home a run, and bringing up Brent Mayne, who worked the count full before he lined to right field for the third out. Olson put together one of the strangest saves imaginable, working around six walks in 1+1⁄3 innings. He threw 49 pitches (not counting the bases-loaded intentional walk) and only 22 of them were for strikes. Olson's only major-league hit was a home run during his last major-league at-bat,[7] on April 20, 1998.[8]
Olson was replaced by new closer Matt Mantei in 1999. He finished his career as a setup man for the Dodgers.
In a 14-year career, Olson compiled 217 saves with a 40–39 record, 588 strikeouts, and a 3.46 ERA in 672 innings pitched.
In 2016, Olson served as pitching coach to actress Kylie Bunbury, who played "Ginny Baker" on the scripted FOX television series Pitch.[10]
From 2017 to 2020 Olson was a color analyst for Orioles broadcasts on MASN.[11] He additionally covered a series for MASN in 2023 when the O's took on the Blue Jays May 19–21.