Appendix
Derivation of a relation between stimulus (effect) size and
response [probability of not requiring further RCTs when\(\mathbf{jnd=}\log\mathbf{(OR)}\mathbf{]}\) based on
analogy to Weber-Fechner law
Let s be the magnitude of a measurable stimulus and Δs the
increase in stimulus just required to discriminate between stimuli
as:21
\begin{equation}
r=\frac{\Delta s}{s}=constant\nonumber \\
\end{equation}This means that the noticeable difference in sensation
occur only when the increase, or change in stimuli (such as change in
the magnitude of effect due to the experimental treatment compared with
control treatment) are a constant percentage of the stimulus
(s ) itself. This is Weber’s law.
Fechner proposed a method of scaling that takes Weber’s law into
account; let \(s_{o}\) be a fixed value of s to allow us to
calculate the nearest noticeably higher stimulus as21:
\begin{equation}
r=\frac{s_{1}-s_{o}}{s_{o}}\ \text{or\ }r\cdot s_{o}=s_{1}-s_{o}\ \nonumber \\
\end{equation}\begin{equation}
s_{1}=s_{o}+r\cdot s_{o}=s_{o}\cdot\left(1+r\right)=s_{o}\cdot q\text{\ where\ }q=1+r\nonumber \\
\end{equation}So the “next” stimulus is q times the previous level of “noticeable”
stimulus. Similarly
\begin{equation}
s_{2}=s_{1}\cdot q=s_{o}\cdot q\cdot q=s_{o}\cdot q^{2}\text{\ \ \ and\ \ }\ s_{3}=s_{2}\cdot q=s_{1}\cdot q^{2}=s_{o}\cdot q^{3}\nonumber \\
\end{equation}which leads to general equation
\begin{equation}
s_{n}=s_{o}\cdot q^{n}\nonumber \\
\end{equation}Which, after taking a natural logarithm (or logarithm base 10) is
analogous to
\begin{equation}
n=A\cdot\ln s+B\nonumber \\
\end{equation}where \(A=\frac{1}{\ln q}\) and \(B=-\frac{\ln s_{o}}{\ln q}\).
In our case, we actually only want to distinguish whether the difference
in stimulus is noticeable, i.e., we are not interested in the absolute
size of the stimulus, but rather whether the ratio \(r\)is large enough.
We express the ratio as effect size measured in terms of odds ratio
(OR), or proportional reduction of OR (1-OR, in terms of reducing bad
events); or, increase in terms of improving good outcomes (OR-1). These
effects are commonly assumed to remain constant over the range of
predicted risks33, providing further justification for
application of Weber-Fechner law:
\begin{equation}
r=\frac{\Delta s}{s}=\frac{ODDSexp-ODDSctrl}{\text{ODDctrl}}=\frac{\text{ODDSexp}}{\text{ODDSctrl}}-1=OR-1\nonumber \\
\end{equation}and
\begin{equation}
q=\left(1+r\right)=\left(1+OR-1\right)=OR\nonumber \\
\end{equation}where ODDSexp and ODDSctrl, are odds of events related to
the experimental and control treatment, respectively.
Rather than using the size of the stimulus (n), we are using the size of
the ratio (\(r\) or \(q\)) as the independent variable. Since this is a
ratio, the use of logarithmic scale is more appropriate, and rather than
modeling the scale or intensity of the stimulus (n) as in the Weber’s
law, we are modeling whether the size of this ratio will indicate a
distinguishable difference :
\begin{equation}
\text{diff\ }\left(\text{yes\ or\ no}\right)=A\cdot\ln\left(q\right)+B=A\cdot\ln\left(\text{OR}\right)+B\nonumber \\
\end{equation}Since in this case, this difference can only take two values
(0=no=further RCTs are required or 1=yes=further RCTs are not required),
we employ logistic regression48 to obtain probability
of not requiring further RCT:
\begin{equation}
\text{logit}\left(P\left(\text{no}n_{\text{RCT}}\right)\right)=A\cdot\ln\left(\text{OR}\right)+B\nonumber \\
\end{equation}Similar expression can be derived using relative risks instead of OR.