Abstract
Background: General practice websites are getting more relevant as more interaction with patients is occurring digitally; however, there are no regulations governing them, nor much research about their benefit or quality.
Objectives: To understand the online presence of general practices through time, using a tool developed for this purpose and assessing practices’ websites at two separate times.
Methods: A form was developed to assess websites on base of their performance against 17 parameters, including information about the practice (logo, interactive address, area covered, telephone, opening times, details about staff, and quality rate), website parameters (number of pages and links on the site, level of link errors, loading speed, and search engine), and online services provided (social media, interactions like messaging, booking appointments, ordering prescriptions, accessing medical records, and allowing video consultations). Possible scores ranged from 17–66 points, with two to four points per factor analysed. A lower score was a mark of quality.
Results: Practices were scored twice over the last 5 years; however, the sample average, although improving from 36.5 to 31.6, still puts practices websites midway in the quality range of the tool used. Looking at individual parameters, the homepage’s average loading time deteriorated, as well as maps showing the practice location. However, other details improved.
Conclusions: Websites are taking a more relevant role in healthcare interactions, but their quality has improved little in the last 5 years. There is a need for stricter regulations if the digital presence of primary care is expected to play a safe role in primary care provision.
Key Points
1. Patients access more and more primary care websites looking for information for their health needs, and to interact with their practices; however, the websites for primary care physicians are quite variable in content and quality.
2. An instrument was created to assess and compare practices’ websites quality through time. When the tool was used in the practices in Leeds, UK, in 2016 and 2021, improvements were found, but not consistently.
3. Guidelines and regulations are needed to improve further the standards of practices’ websites, so the digital presence of general practitioners provides good quality and safe advice and interactions with their patients.
INTRODUCTION
General practices in the UK are under immense pressure due to numerous factors and the role of health informatics. Research has been conducted on subjects such as the complexity of current consultations1 and the use of electronic consultations.2,3 Unfortunately, their websites and the quality of their digital presence (the way the organisation appears and interacts on the internet) has not received the same attention; studies are more limited4 or simply old.5 It seems that when the digitalisation of primary care was taking place, some interest in the matter was present; however, this is no longer the case. Even the latest long-term plan from the National Health Service (NHS) fails to mention any requirements regarding practices’ websites when exploring digital solutions use.6 There are no current guidelines or regulations on health information on websites, and practices simply subcontract to website developers offering their solutions. With the COVID-19 pandemic and an accelerated shift towards digital working, the quality of this important portal of health information becomes even more relevant.
It is not possible to reduce primary care physicians’ workload without educating patients to look after themselves, providing them with reliable sources of information, and signposting them to other services that could be more convenient for their needs. For this purpose, the practice website could provide invaluable information. Equally, practices must share information about themselves, the services they offer, their team, and the quality of service provided, as per regulators assessment. Patients interacting with the website and searching for answers to their needs should receive an appropriate level of quality information.
The ability to assess a single primary care website against a set of parameters can facilitate an understanding of their quality of service. An easy to use but comprehensive assessment tool for primary care websites could facilitate the process and allow for comparisons among organisations. Furthermore, its use at different times could provide light on their quality trend.
METHODS
This project consisted of two steps: the creation of an easy-to-use scoring instrument and its implementation on a sample of practices.
Short Website Assessment Tool in Primary Care
For the assessment tool, the parameters used were ones that had been previously considered for evaluation of general practice websites; one based in the UK with the STaRNet Website Assessment Tool (SWAT)5 and the other in the USA.4 Unfortunately, it seems that this subject has been absent in the medical literature for quite some time, and no relevant references were found when a literature search was conducted looking for “website” and “primary care.” The parameters from these papers were evaluated, and some were added from current English recommendations. Some were numeric (such as the number of links on the site), some were not (such as the presence of opening hours). Eventually, 17 features contributed to the score. The better the performance, the lower the score in each of the factors assessed. The SWAT in primary care tool responses followed a strict protocol, with “yes;” “no;” and some “partial” answer options, as the information could be present in a different format, or location than the ideal. The numeric values obtained were later added into well-defined categories to facilitate scoring.
The assessment was divided into three sections, and the parameters and reasoning are as follows.
Information about the practice
One basic role of a practices’ website is to be a virtual brochure of the primary care organisation,7 and practices in England are expected to have an “informative online presence.”8 It is also relevant to consider there are additional mandatory requirements, including delivering patient online services,9 and their quality rating according to the Care Quality Commission (CQC).10
Logo
Organisations are expected to have a unique name and logo.11 General practitioners should have their own identity that is clearly shown and not just the NHS logo.
Address with interactive map
Practice addresses are important to consumers12 and are part of the minimum information requirements about practices.7 Furthermore, an interactive map, a powerful tool to facilitate locating the place and getting directions, is paramount. As a minimum, some sort of map should be present to facilitate locating the physical location of the practice.
Practice area with interactive map
It is mandatory in the UK for practices to indicate the area from which patients are allowed to register.7 This information should be easily available on the website for patients to consult. However, as above, it should consist of a responsive tool that facilitates finding out if a patient can or cannot register within that particular practice, considering the personal address of the potential patient related to the practice address. Any other form of a map or textual explanation should be considered the minimum.
Telephone number (on the homepage)
It is common to use the Internet to find contact numbers and, ideally, the contact details should be visible on the homepage, without the need to navigate to find it. Furthermore, its digital presence is a minimum requirement.7
Opening hours (on the homepage)
If the website has been used to find how to communicate with the practice, it would be expected that contact would follow and, for that purpose, an idea of when the premises are open for business is paramount. It is also a minimum requirement.7
Doctors’ names, qualifications, and pictures
Clinician names and their qualifications are mandatory on practice leaflets.7 For transparency, clinicians’ portraits also should be present. With staff changes, it could be that not all details are present, but at least half of the clinicians’ details should be present in full to consider a level of quality and commitment to keep the site up to date.
Nurses and non-clinical staff names and pictures
All members of staff should be identifiable on the website.
Practice rating (on the homepage)
Ratings are a way to assess the likelihood that the organisation will provide the services intended at a certain standard. In the UK, practices must be registered under the CQC and show their rating.10 Ideally, it should be on the homepage and quite visible.
Website parameters
Looking into the more technical detail of the websites, a free online tool called Dead Link Checker (Petersfield, UK) was used to assess how many links were present, and how many were not functional.13 A well-maintained practice website should have many links with a little number of rotten ones.
Number of homepage links of >100
It is difficult to determine how many pages a medical practice website should contain. It is clear that it should not recreate information elsewhere, but to signpost patients to good quality sources. As a consequence, the assessment was based on links, including on the homepage. Links to pages within the site as well as external sites were included; however, they were also considered to be elements of the page.
Number of website links of >500
As with the above, it is difficult to quantify what number of links makes a primary care website one that is of a good quality. In the past links, to medical websites, medical journals, support groups, and more were considered.4,5 For the purpose of this tool, the assumption is that a reasonable number of links would be required for the basic signposting role of the website, and it is stated the need to have over 500 links.
Percentage of link errors
The internet is not static and ‘link rot’ is common, affecting not only electronic citations14 in the literature but everything. Organisations should keep links up to date. A quality site should not have more than a handful of broken links. As the sites can be quite different in the number of links available, it is more reliable to have the percentage of link errors related to the number of links present on the site. Having <1% of ‘rotten’ links was considered a sign of excellent quality.
Loading speed
Information should be provided without a long wait, and the homepage should load quickly. For this assessment, load times were calculated using an automation tool, Selenium (Thoughtworks, Chicago, Illinois, USA),15 which loaded the websites’ unique uniform resource locators in a headless browser. The resulting time would be the difference between ‘navigationStart’ and ‘domComplete’, which should be recorded 5 times per website to obtain an average.
All recordings need to be taken with the browser running in ‘no cache’ mode, ensuring that no requests were cached, as this could potentially affect the loading times between retries. The practices were scored between 1 (best) and 4 (worst). Websites that used caching and took <500 ms to load on average were given a score of 1. Meanwhile, websites that took 500–800 ms per page were given a score of 2, and a score of 3 given to those taking 801–1,200 ms. Finally, a 4 given to websites where the average page took >1.2 seconds.
Search engine
They significantly increase a website’s usability, and they are expected to be present nowadays on any website.
Online services
Websites are not just a portal for customers to get information from, but a point of interaction and business. In healthcare, this could be through associated social media pages, messaging capabilities, and signposting to online lifestyle and health management sources; however, these should be via specific online services such as the already mentioned requisites of appointments, prescriptions, and electronic records access.7
Social media presence
It is clear that social media can facilitate health promotion, although it is also behind health misinformation.16 Primary care should contribute to sharing reliable information on these platforms, and to balance the information available from less reputable sources that can undermine the health of the patients. Practices should be actively using social media.
Messaging capabilities
Websites need to provide this functionality, whether by email or non-clinical website forms, so that communication between patient and practice can take place digitally.
Core online services (appointments, prescriptions, and records)
All practices are expected to allow these services7 through their clinical software. The website should provide a link to access the patient’s electronic health record (EHR). Additional clinical website forms or e-consultations could be offered.
Use of video consultations
According to the agreed framework between NHS England and the British medical Association (BMA): “All patients will have the right to online and video consultations by April 2021.”8 In consequence, practices are expected to promote the service on their websites, and to indicate how to access this type of consultation.
Scoring
Finally, there was the question of whether there was a risk of variability to scoring. Data to qualify for each particular score was quite clear, so that inter-rater reliability and repeatability testing should not be required. The best possible score would be 17 and the worst possible would be 66 (Table 1).
Table 1: Tool scoring card.
CQC: Care Quality Commission; EHR: electronic health record; NS: no score; px: prescriptions.
Practices Websites Identification
General practitioners and their websites in the city of Leeds, which was divided into 94 primary care organisations in 2016, were chosen, as they were local and known to authors. The authors also considered that the size of the sample should be sufficient to come to relevant conclusions. They were identified using data available from the then three Leeds clinical commissioning groups.
Although there are clear changes in the landscape of primary care in Leeds. The three clinical commissioning groups had merged into one and there were other mergers and closures of some health centres, as well as changes in website portal names. The cohort of the organisation/websites for the follow up were identified, and it had been reduced to 81 sites.
RESULTS
Several issues can be defined when analysing the data obtained, such as the overall scores and the parameter changes between the two years in which the data was collected. A summary of findings is presented in Figure 1 and Table 2.
Figure 1: Cumulative scores in 2016 (A) and 2021 (B) among the practices for the 17 parameters assessed.
CQC: Care Quality Commission; px: prescriptions.
Table 2: Average values per parameter and total.
CQC: Care Quality Commission.
General Scores
In 2016, among the 94 organisations, the average score was 36.5, with a minimum of 27.0 and a maximum of 49.0 (standard deviation: 3.8). In 2021, the average score was 31.6 among the remaining 81 enterprises, with a minimum of 21.0 and a maximum of 40.0 (standard deviation: 3.8).
No statistical analysis is present, as there have been changes in the number of organisations and only descriptive statistics were used as the number of names and names of organisations changes in the 5-year period of the study. The aim was to present a general picture of the evolving and current presence in a location that could be extrapolated to understand primary care presence in the UK.
Particular Parameters
Considering the number of organisations in each sample is different, data are presented as percentages. Statistical differences among practices are not presented, as it is not the aim of this study, and the sample was small.
Logo
Logos were used by 37.2% of organisations in 2016, increasing to 43.2% in 2021.
Address with interactive map
An address with an interactive map was present in 85.1% in 2016; however, it had declined to 78.7% in 2021. Some practices were simply putting a picture of a map or free text rather than a helpful interactive map to locate their premises.
Practice area with interactive map
Catchment area was not so commonly linked to an interactive map, used on only 20.2% of practices’ website in 2016; however, its use increased considerably to 40.4% in 2021.
Telephone number (within the homepage)
The presence of a telephone number on the homepage was almost unanimous in 2016 (95.7%), with the only 4 organisations having it on a different page on the website; however, it declined in 2021 to 87.7%, with 10 practices offering the details elsewhere.
Opening hours (within on the homepage)
Opening hours was present on the homepages in 26.6% of websites and absent in 1.1% in 2016. in contrast, its occurrence in the homepages increased considerably to 59.3% in 2021, and if not, it was present on another page.
Doctors’ names, qualifications, and pictures
Doctors’ names, qualifications, and portraits were present in 73.4% of websites in 2016 and absent in only 3.2% of practices websites. However, a decline was noted in 2021, with 58.0% of websites adequately recording this data and 6.2% not having it available to patients.
Nurses and non-clinical staff names and pictures
In 2016, in 53.2% of website sites had an appropriate number of details about nurses and non-clinical staff members; however, these were absent in 6.4% of websites. In 2021, it was similar as 56.8% of practices included these details while they were absent in in 13.6% of websites.
Practice rating (on the homepage)
A CQC rating was present on the homepages of 28.7% of websites in 2016 and absent in 61.7%; however, it was very different in 2021, where the CQC rating was present on the homepages of 80.2% of website and elsewhere on the website in the other cases..
Number of homepage links of >100
The median value for the number of homepage links in 2016 was 96. It varied between one and 500. The maximum was determined by the link checker used, affecting the score in nine website portals in 2016 and none in the second group. In 2021, it varied between three and 237, with a median of 158 and an average of 143 links per site. For scoring, three groups were created, depending on the number of links being higher than 100, between 50–100, and below 50. In 2016, 43.6% of practices sites had >100 links, and in 2021 that increased to 77.8%.
Number of website links of >500
The median number of links in 2016 was 780, ranging from 10–2,000. This was limited by the link checker used, and affecting eight websites in 2016 and three websites 5 years later. In 2021, the median was 966, ranging from 3–2000. When considered in groups (more than 500, between 100–500, and <100 links), 79.8% of sites had more than 500 links in 2016 and that percentage was 88.9% in 2021.
Percentage of link errors
Link errors in 2016 ranged from 2–822, with a median of 16. In 2021, it ranged between 0–953, with a median of 26. The authors considered that the ideal is to have <1% of links broken. This was the case in 9.6% of websites in 2016, while it was 11.1% in 2021.
Loading speed
Websites were typically not optimised for speed; not utilising content delivery networks effectively; and, in some cases, downloaded excessively large content. It was found that most of the websites utilised caching effectively, with most sites having cascading style sheets/images cached to reduce load time in subsequent uploads. This typically reduced loading times between 50–70%. Page loading speed in 2016 was best (<500 ms) in 4.3% of cases, while it was worst (>1.2 secs) in 9.6% of websites. In contrast, there was a general deterioration in 2021, even though 6.2% scored best as there was an increase in the number of websites that were classed as worst (an increase of 55.6%). When considering this aspect, one outlier should be mentioned: a homepage that took over 8 secs to download as there was a video embedded within it, a 42.3 MB file. The direct result was that the website was non-operational, with no links accessible, until the download was complete.
Search engine
A search engine was present in 84% of sites analysed in 2016, and in 92.6% in 2021.
Social media presence
Social media presence was uncommon in 2016; it was found in 14.9% of practices’ websites, and it is still only in 25.9% in 2021.
Messaging capabilities
Messaging capabilities (email address and forms on website to communicate non-clinical issues) was provided by 76.6% of practices in 2016, and it improved in 2021 to 79.0%.
Core online services (appointments, prescriptions, and records)
Core online services, such as access to hers and additional services in the way of e-consultations, were present in 51.1% of practices in 2016. However, there were no links to EHR in 43.6% of cases. In contrast, all practices offered online access to her in 2021, and also provided access to additional services in 64.2% of websites.
Use of video consultation
Video consultations were only advertised by 3.2% of practices in 2016; however, the percentage increased to 19.8% in 2021.
DISCUSSION
New regulations have been implemented in Europe regarding medical devices, defined as “products or equipment intended for a medical purpose.”17 Medical websites and social media, where there is an interaction among patients and clinicians through software, is not yet considered one. No regulation applies to this digital platform, even though its importance as a communication tool is increasing. Furthermore, advice can be given for symptoms without direct clinician involvement, whether for the triage of urgency for appointments or to self-management, for example. Therefore, it is important to start assessing the quality of the internet presence of general practice and its interaction with the patients, and how can it potentially affect outcomes. This paper has presented the variability in some basic quality markers among different organisations and has suggested some standards. The situation is further complicated the more the website responds automatically to patients’ input.
A website from a primary care physician can be quite limited in its content but still can have a considerable impact on a patient looking for health information. There is a need to regulate them, improve standards, and improve their safety as more and more patients use them, particularly since the recent COVID-19 pandemic.
The assessment completed among these organisations showed that there is fluctuation in the current unregulated landscape, and not necessarily improvement when data 5 years apart are compared.
The descriptive statistics used help to highlight that this is an area in urgent need of attention, as currently there are no regulations or standards to comply with. In consequence, primary care online resources are not necessarily safe.
Summary
The tool created was simple to use, and it allowed easy comparisons among different primary care websites on some minimum standards. The maximum score was 66 and the minimum was 17, with 49 points to play, and the higher values indicated poorer quality. A site could be classed as with very good quality if it scored less than 70% of the available points.The smaller the score, the better the resource. So, sites could be classified into three groups based on their scores: very good (<32 points), good (between 32–51 points), and poor (a score above 51 points). This assumption would mean that 12.8% of websites in Leeds were good quality in 2016, while 87.2% were good quality in 2021. No website was classed as poor. The scores varied between 27.0 and 49.0, with an average score of 36.5. When the practices’ websites were reassessed in 2021, 43.2% had very good quality and 56.8% had good standards.
Looking in more detail, some parameters improved while others deteriorated (such the time it took the homepage to load and the location of the practice’s telephone number). There was a limited effect on the overall score. However, the scores provide some insight on areas where practices should consider focusing when aiming to improve the quality of their services.
Strengths and Limitations
Websites analysis is a new subject, and it goes without the attention it requires as the flow of health information moves towards the internet. Practices’ websites need to deliver high standards to attract viewers; however, without clear guidance and probably regulations, service providers are free to offer packages of limited value. There is no recent literature on the matter, and, in consequence, this novel study highlights some of the areas that should be considered, as well as ways of comparing websites.
The tool is limited by its focus, and by the data collected. It has been a study of primary care physicians’ institutions in Leeds. It means some modifications will be required to be used in assessing other healthcare establishments in the UK or other countries. Analysing particular questionnaires and algorithms should require further in-depth studies. However, this tool is nonetheless a starting point to view and compare healthcare centres.
Implications for Research and/or Practice
There is a need to improve websites in primary care and equally a need for clear guidelines, as well as regulations on what should be the minimum requirements for practices, so providers can follow them and improve the current standards of websites, such as the ones assessed in this paper, which below the ideal. Further research could be required on patients’ views on what they would like to see when accessing their practices’ websites, influencing their requirements, and ultimately improving the patient experience. Once the basics have been regulated, further research on the additional risks and benefits of automatic responses from websites triaging or guiding patients will need further research.
CONCLUSIONS
There has been little improvement in the quality of websites over the period of 5 years, with most individual parameters improving while others had deteriorated. Stricter regulations are needed on what is expected from websites if the digital presence of primary care is going to play a relevant role in primary care provision, and if it aims to provide safe advice to patients.
