Keywords
SWAT, Study Within A Trial, attrition, SMS, text messages
This article is included in the Studies Within A Trial (SWAT) collection.
SWAT, Study Within A Trial, attrition, SMS, text messages
We have added additional detail to the article in response to the issues raised by the reviewer. We have outlined how the validity of the phone numbers was checked at two different stages, and also provided detail on the pre-planned retention strategies.
We have also made a minor amendment to the results in light of a duplicate randomisation that was found in the host trial, which means that 2334 rather than 2335 participants were randomised to the host trial. This duplicate randomisation was not randomised to the SWAT and therefore has no other impact on the results. We have updated the figure to take this change into account.
See the authors' detailed response to the review by Michelle E. Kho
Clinical trialists have identified the recruitment and retention of participants as key issues for randomised controlled trials (RCT)1,2.
Several studies have investigated whether personalising trial documentation can aid recruitment and retention3,4. Recently, Cochrane et al. looked at the effect of personalised text messages compared to standard text messages in improving retention rates5. This study was carried out in response to a number of embedded trials evaluating the effectiveness of SMS messages in improving retention rates6–11, alongside a study suggesting personalised messages increased the payment of delinquent fines12.
To further add to the evidence on the effectiveness of personalised text messages, we did a ‘study within a trial’ (SWAT) evaluating the effectiveness of a personalised text message compared to a non-personalised text message on postal questionnaire response rates in a large orthopaedic trial.
This paper details the methods and results of a SWAT embedded within the prospectively registered Knee Replacement Bandaging Study (KReBS) RCT (ISRCTN87127065, registered on 20 February 2017). KReBS evaluated the effectiveness of a two-layer compression bandage compared with a standard wool and crepe bandage applied post-operatively on patient-reported outcomes in total knee replacement patients13.
The SWAT was conducted in 26 NHS hospital trust sites and was implemented at the start of the study. All KReBS participants were eligible for this SWAT provided they had opted in to receiving SMS messages and were not deceased or withdrawn from follow-up before being due to be sent their 12-month postal questionnaire.
Participants in the SWAT were sent either a personalised or non-personalised text message (Table 1) four days after their 12-month questionnaire was sent.
The validity of the phone number given by participants was checked at two different stages. At the first stage, after randomisation, invalid numbers not beginning with “07”, which are the first two digits of all UK mobile phone numbers were removed from the database, participants providing numbers such as these were not sent a text message. The second stage occurred at the point of the text messages being sent out. Some participants provided phone numbers that began with “07”, but were actually invalid and therefore not picked up at the first stage. An attempt to send a text message to the phone number would have resulted in a failure in the message being delivered.
All SWAT participants received pre-planned retention strategies within KReBS. This consisted of a reminder letter and additional copy of the questionnaire if the participant had not returned a completed copy 4 weeks after sending out the original copy. If there was still no response following the postal reminder, participants were contacted by telephone to obtain the patient-reported outcomes.
The primary outcome was the proportion of participants who returned a 12-month questionnaire. Secondary outcomes were the proportion of participants who completed the questionnaire and time to questionnaire return. A questionnaire was considered complete if the participant had answered 11 or more questions of the 12-item host trial primary outcome, the Oxford Knee Score14.
Since this was an embedded trial, the sample size was determined by the number of participants in the main KReBS trial13, which aimed to recruit 2600 participants.
Participants were randomised into the SWAT using simple randomisation in a 1:1 allocation ratio. The allocation schedule was generated by a researcher at the York Trials Unit not involved in the recruitment or follow-up of participants.
Analyses were carried out using Stata v16.015. A diagram detailing the flow of participants through the SWAT is provided, and baseline characteristics are presented by SWAT allocation. Outcomes are summarised descriptively. Statistical tests were two-sided using a 5% significance level, and were done on an intention to treat basis. All analyses (except the calculation of the absolute difference in return rate which was estimated using the two-sample test of proportions) used mixed effects regression, adjusting for SWAT allocation and host trial allocation as fixed effects and trial site as a random effect. Relevant parameter estimates are presented with associated 95% confidence intervals and p-values.
The proportion of participants who returned a 12-month questionnaire, and proportion complete, was analysed using logistic regression. A second SWAT evaluating receipt of a pen on response rates was also embedded in KReBS at 12 months16. In a sensitivity analysis, we additionally adjusted the primary model for pen SWAT allocation.
Time to questionnaire return was analysed using a Cox proportional hazards shared frailty model. Participants who did not return a questionnaire were censored at 90 days.
In total, 2334 participants were recruited into the KReBS trial and 1470 were randomised to the SWAT (Figure 1). The average age was 66.8 years and 54.0% were female (Table 217). Five participants died or withdrew following randomisation and as a result 723 participants in the personalised group, and 742 in the non-personalised group, were sent a 12-month questionnaire and were included in the analysis. Of these, 680 (94.1%) of the 723 participants in the personalised group, and 701 (94.5%) of the 742 in the non-personalised group, were sent a text.
In the personalised group, 644/723 (89.1%) participants returned a questionnaire, compared to 654/742 (88.1%) in the non-personalised group (Table 317). The absolute difference in return rate was 0.9% (95% CI: -2.3% to 4.2%; p=0.57). There was no evidence of a difference between the groups in the likelihood of returning a questionnaire (OR 1.09; 95% CI: 0.79 to 1.51; p=0.61), the likelihood of returning a complete questionnaire (OR 1.11; 95% CI: 0.82 to 1.51; p=0.50) nor in time to return (HR 1.05; 95% CI: 0.94 to 1.17; p=0.40). In total, 1465 participants were also randomised to the pen SWAT. When the primary model was repeated with the addition of pen SWAT allocation, the results remained the same.
This embedded trial found little evidence to suggest personalised text messages are more effective than non-personalised text messages in encouraging return and completion of questionnaires. The trial did not find evidence of a statistically significant difference between groups in any of the outcomes, although effect size estimates favoured the personalised group. On the other hand, while Cochrane and colleagues also did not find evidence of a statistically significant difference between groups, estimates of effect mostly favoured the non-personalised group5.
The SWAT had a large sample size, which means the results can be generalised to other orthopaedic studies. However, completion rate was calculated as a proportion of all SWAT participants rather than all SWAT participants who returned a questionnaire, and as a result questionnaire completion was highly correlated with questionnaire return. In addition, some participants included in the analysis did not receive a text message.
This SWAT adds to the growing evidence base for whether personalised trial documentation, in particular text messages, are effective.
Open Science Framework: Underlying data and CONSORT diagram for an embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting. https://doi.org/10.17605/OSF.IO/KHJ8E17
This project contains the following underlying data:
Open Science Framework: CONSORT checklist for ‘An embedded randomised controlled retention trial of personalised text messages compared to non-personalised text messages in an orthopaedic setting’ https://doi.org/10.17605/OSF.IO/KHJ8E17
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
Partly
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Trauma, Orthopaedic and Rehabilitation Clinical Trials
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
References
1. Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, et al.: Methods to increase response to postal and electronic questionnaires.Cochrane Database Syst Rev. 2009. MR000008 PubMed Abstract | Publisher Full TextCompeting Interests: No competing interests were disclosed.
Reviewer Expertise: Research methodology, trial design and reporting
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 2 (revision) 15 Sep 21 |
read | |
Version 1 11 Jun 20 |
read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)