Abstract It is known that there exist obivious differences between the two most commonly used definitions of fractional derivatives—Riemann–Liouville (R–L) definition and Caputo definition. The multiple definitions of fractional derivatives in fractional calculus have hindered the application of fractional calculus in rheology. In this paper, we clarify that the R–L definition and Caputo definition are both rheologically imperfect with the help of mechanical analogues of the fractional element model (Scott–Blair model). We also clarify that to make them perfect rheologically, the lower terminals of both definitions should be put to -∞. We further prove that the R–L definition with lower terminal a → -∞ and the Caputo definition with lower terminal a → -∞ are equivalent in the differentiation of functions that are smooth enough and functions that have finite number of singular points. Thus we can define the fractional derivatives in rheology as the R–L derivatives with lower terminal a → -∞ (or, equivalently, the Caputo derivatives with lower terminal a → -∞) not only for steady-state processes, but also for transient processes. Based on the above definition, the problems of composition rules of fractional operators and the initial conditions for fractional differential equations are discussed, respectively. As an example we study a fractional oscillator with Scott–Blair model and give an exact solution of this equation under given initial conditions.