research papers on wireless technology

5G wireless technology evolution: identifying evolution pathways of core technologies based on patent networks

• Published: 28 October 2023

Cite this article

• Biqiang Han 1 ,
• Jie Zhang 2 ,
• Helen Cai 3 ,
• Mengyao Xia 4 ,
• Yan Tu 5 &
• Jiao Wu 6  

195 Accesses

Explore all metrics

With the rapid development of 5G technology, studying its evolution path is crucial for innovation and development. This study explores the evolution of 5G technology using patent data analysis. By applying main path analysis and network techniques to a dataset from the IncoPat patent database, the study identifies the main path of 5G technology evolution and analyzes its trends. The Citespace is used to construct citation networks and reveal the main path of the patent citation network, regional citation network, and institutional citation network. Furthermore, the study incorporates the 5G technology lifecycle to dynamically interpret the main path and analyze the major evolution trends of 5G technology. The methodology includes data collection from the IncoPat patent database and the adoption of clustering analysis to identify community structure within citation networks. The Log Likelihood Ratio (LLR) algorithm is employed for community detection, facilitating the analysis of interrelationships among different communities. The main path analysis involves traversal counting and path searching, with the LLR algorithm selected to identify critical nodes in the evolutionary paths of technologies. Additionally, dynamic main path analysis is conducted by incorporating the temporal attribute, enabling the observation of the dynamic changes in technological evolution. The study concludes with important insights into the current development status and evolution of 5G technology, shedding light on key paths and trends in 5G technology evolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Technological Trajectory Analysis of Patent Citation Networks: Examining the Technological Evolution of Computer Graphic Processing Systems

Ichiro Watanabe & Soichiro Takagi

Consistency and Trends of Technological Innovations: A Network Approach to the International Patent Classification Data

A hybrid approach to detecting technological recombination based on text mining and patent network analysis

Xiao Zhou, Lu Huang, … Miaomiao Yu

Data Availability

The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.

Ahonen, T. (2009). Mobile as the 7th of the Mass Media. In Mobile as 7th of the Mass Media (pp. 3–21). Forum Oxford.

Boccardi, F., Heath, R. W. Jr., Lozano, A., Marzetta, T. L., & Popovski, P. (2014). Five disruptive technology directions for 5G. IEEE Communications Magazine , 52 (2), 74–80.

Article   Google Scholar  

Kim, J., Lee, S. H., Jeong, D. G., Kim, S. W., Kim, J. H., & Lee, Y. H. (2017). 5G communication: Evolution and requirements. Journal of Electrical Engineering and Technology , 12 (2), 589–595.

Google Scholar  

Li, C., Qiu, D., Li, B., & Li, G. (2017). 5G wireless communications: Based on existing waveforms or new waveform design? IEEE Communications Magazine , 55 (2), 109–115.

Rappaport, T. S., Sun, S., Mayzus, R., Zhao, H., Azar, Y., Wang, K., & Shafi, M. (2013). Millimeter wave mobile communications for 5G cellular: It will work! Ieee Access : Practical Innovations, Open Solutions , 1 , 335–349.

Chen, H., Zhao, Y., Zhang, X., & Zhang, J. (2016). Research on key technologies and open issues of 5G networks. Ieee Network , 30 (4), 44–51.

Gudipati, S., & Ratasuk, R. (2017). The journey of 5G: Standards, patents, and the future of mobile broadband. IEEE Communications Magazine , 55 (1), 176–183.

Ji, Y., Zhang, R., Zhang, T., & Liu, A. (2017). 5G mobile communication networks: A review. Journal of Zhejiang University Science C , 18 (3), 243–270.

Osseiran, A., Boccardi, F., Braun, V., Kusume, K., Marsch, P., Maternia, M., & Tullberg, H. (2014). Scenarios for 5G mobile and wireless communications: The vision of the METIS project. IEEE Communications Magazine , 52 (5), 26–35.

Giordani, M., Mahmoodi, T., & Tafazolli, R. (2017). Towards 5G tactile internet: Characterizing the human touch. IEEE Communications Magazine , 55 (2), 116–123.

Khan, A. U., Ahmed, M., Alnuem, M. A., Ghoneim, A., & Kadir, M. A. (2018). 5G network slicing using SDN and NFV: A survey of taxonomy, architectures and future challenges. Telecommunication Systems , 69 (3), 389–416.

Yang, Y., Zhang, L., Zhang, Q., Wang, X., & Wang, K. (2019). Towards intelligent 5G: AI-driven resource allocation and optimization. Ieee Network , 33 (2), 70–76.

Chen, J., Zhang, J., Wei, H., & Dai, J. (2023). Frontiers in web-based energy management research: A scientometric data report. Frontiers in Energy Research , 11 , 1195243.

Bai, Y., Chou, L., & Zhang, W. (2021). Industrial innovation characteristics and spatial differentiation of smart grid technology in China based on patent mining. Journal of Energy Storage , 43 , 103289.

Breschi, S., Lissoni, F., & Malerba, F. (2003). Knowledge-relatedness in firm technological diversification. Research Policy , 32 (1), 69–87.

Tang, S., Cai, H.H., Xia, M., Turi, A.N. (2023). The Surge in Blockchain-Based Patent Applications: Booster or Bumps?. In: Turi, A.N. (eds) Financial Technologies and DeFi. Financial Innovation and Technology. Springer, Cham. https://doi.org/10.1007/978-3-031-17998-3_8

Hall, B. H., Jaffe, A. B., & Trajtenberg, M. (2005). Market value and patent citations. The RAND Journal of Economics , 36 (1), 16–38.

Li, J., Li, X., Zhang, S., & Zhu, D. (2021). Research on Evolutionary Path of Automobile Technology Based on Patent Data: A Case Study of Chinese Automobile Industry. Complexity, 2021, 6636039.

Zhang, M., Chen, S., & Li, J. (2021). Exploring the evolutionary path of nanotechnology: A patent analysis based on Technology Co-occurrence Network. Journal of Cleaner Production , 312 , 127720.

Zhang, Y., Li, G., Yan, Y., Li, W., & Wang, Z. (2021). A patent analysis Method based on hierarchical latent Dirichlet Allocation and Association Rule Network. Ieee Access : Practical Innovations, Open Solutions , 9 , 16169–16182.

Li, X., Yin, H., & Yang, L. (2022). Analyzing the evolution path of Artificial Intelligence Technology: A dynamic topic modeling Approach. Ieee Access : Practical Innovations, Open Solutions , 10 , 6480–6492.

Zhang, J., Chen, Y., Wei, Y., & Wang, Y. (2021). Technology evolution path analysis based on Technology Entropy Tree: A case study of IT Technology. Journal of Systems Science and Information , 9 (5), 59–76.

Qiu, H. H.,Liu LG.A study on the evolution of carbon capture and storage technology based on knowledge mapping[J].Energies,2018,11(5):1103–1104.

Liu, J., Meng, Z., & Jiang, Z. (2017). Synergistic Innovation Research of element-structure-function-cost based on S-Curve. Science and Technology Management Research , 37 (16), 146–151.

Zhu, H. (2021). Analysis of 5G Technology Evolution Path from the Perspective of Patent Citation (Master’s thesis, Anhui University of Finance and Economics).

Dahesh, M. B., Tabarsa, G., Zandieh, M., & Hamidizadeh, M. (2020). Reviewing the intellectual structure and evolution of the innovation systems approach: A social network analysis. Technology in Society , 63 , 101399.

Narin, F., Hamilton, K. S., & Olivastro, D. (1997). The increasing linkage between US technology and public science. Research Policy , 26 (3), 317–330.

Zhang, Y., Chen, X., & Wu, Y. (2020). Identifying technology evolution pathways: A bibliometric analysis based on the main path analysis and patent citation network. Technological Forecasting and Social Change , 151 , 119791.

Wang, C., Zhang, X., & Li, Y. (2020). Evolutionary trajectory and path dependency of technological innovation in industrial robot industry: A main path analysis. Technological Forecasting and Social Change , 150 , 119756.

Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. " MIS Quarterly , 36 (4), 1165–1188.

Leydesdorff, L., & Rafols, I. (2009). A global map of science based on the ISI subject categories. Journal of the American Society for Information Science and Technology , 60 (2), 348–362.

Wang, X., Chen, H., & Chen, X. (2021). Identifying Emerging Technologies: A text Mining Approach. Journal of Informetrics , 15 (1), 101157.

Trajtenberg, M. (1990). A penny for your quotes: Patent citations and the value of innovations. The RAND Journal of Economics , 21 (1), 172–187.

Dangi, R., Lalwani, P., Choudhary, G., You, I., & Pau, G. (2021). Study and investigation on 5G technology: A systematic review. Sensors (Basel, Switzerland) , 22 (1), 26.

Janssens, F., & Huysmans, J. (2019). Unveiling the history of science using citation network analysis: Implementing the dynamic topic model (DTM) on the entire citation network of Nature. Scientometrics , 118 (2), 513–538.

Dunning, T. (1993). Accurate methods for the statistics of surprise and coincidence. Computational Linguistics , 19 (1), 61–74.

Chen, C. (2006). CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology , 57 (3), 359–377.

Brandes, U., & Corman, S. R. (2003). Visual unrolling of network evolution and the analysis of dynamic discourse. Journal of Social Structure , 4 (2), 1–29.

Batagelj, V., & Mrvar, A. (2002). Pajek: Program for large network analysis. Connections , 24 (2), 47–57.

MATH   Google Scholar  

Narayanan, A., Ramadan, E., Carpenter, J., Liu, Q., Liu, Y., Qian, F., & Zhang, Z. L. (2020, April). A first look at commercial 5G performance on smartphones. In Proceedings of The Web Conference 2020 (pp. 894–905).

Banaseka, F. K., Katsriku, F., Abdulai, J. D., Adu-Manu, K. S., & Engmann, F. N. A. (2021). Signal propagation models in soil medium for the study of wireless underground sensor networks: A review of current trends. Wireless Communications and Mobile Computing , 2021 , 1–12.

Sodhro, A. H., Obaidat, M. S., Abbasi, Q. H., Pace, P., Pirbhulal, S., Fortino, G.,… Qaraqe, M. (2019). Quality of service optimization in an IoT-driven intelligent transportation system. IEEE Wireless Communications , 26 (6), 10–17.

Download references

This study was supported by the Shaoxing City Philosophy and Social Science Research ‘14th Five-Year’ Plan 2023 Key Project: A Study on the Implementation Path of Rural Revitalization Empowered by Digital Transformation under the ‘Two Mountains’ Theory from the Perspective of ‘Green Common Prosperity’ – A Case Study of Zhejiang Shaoxing (Project Number: 145J073)

Author information

Authors and affiliations.

Research and Local Cooperation Office, Wenzhou Business College, Wenzhou, China

Biqiang Han

School of Economics and Management, Shaoxing University Yuanpei College, Shaoxing, China

Business School, Middlesex University, London, UK

School of Business, Nanjing University of Information Science & Technology, Nanjing, China

Mengyao Xia

School of Arts, Zhengzhou University of Science and Technology, Zhengzhou, China

School of Agricultural Science and Engineering, Hohai University, Nanjing, China

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jie Zhang .

Ethics declarations

Conflict of interest.

Author Biqiang Han declares that she has no conflict of interest. Author Helen Huifen Cai declares that he has no conflict of interest. Author Mengyao Xia declares that he has no conflict of interest. Author Jie Zhang declares that he has no conflict of interest. Author Yan Tu declares that he has no conflict of interest. Author Jiao Wu declares that she has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Not Applicable.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Han, B., Zhang, J., Cai, H. et al. 5G wireless technology evolution: identifying evolution pathways of core technologies based on patent networks. Wireless Netw (2023). https://doi.org/10.1007/s11276-023-03538-8

Download citation

Accepted : 22 September 2023

Published : 28 October 2023

DOI : https://doi.org/10.1007/s11276-023-03538-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Intelligence
• Wireless Technology
• Network Analysis
• Find a journal
• Publish with us
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 12 January 2021

5G as a wireless power grid

• Aline Eid 1 ,
• Jimmy G. D. Hester 1 , 2 &
• Manos M. Tentzeris 1  

Scientific Reports volume  11 , Article number:  636 ( 2021 ) Cite this article

125k Accesses

50 Citations

395 Altmetric

Metrics details

• Devices for energy harvesting
• Electrical and electronic engineering

5G has been designed for blazing fast and low-latency communications. To do so, mm-wave frequencies were adopted and allowed unprecedently high radiated power densities by the FCC. Unknowingly, the architects of 5G have, thereby, created a wireless power grid capable of powering devices at ranges far exceeding the capabilities of any existing technologies. However, this potential could only be realized if a fundamental trade-off in wireless energy harvesting could be circumvented. Here, we propose a solution that breaks the usual paradigm, imprisoned in the trade-off between rectenna angular coverage and turn-on sensitivity. The concept relies on the implementation of a Rotman lens between the antennas and the rectifiers. The printed, flexible mm-wave lens allows robust and bending-resilient operation over more than 20 GHz of gain and angular bandwidths. Antenna sub-arrays, rectifiers and DC combiners are then added to the structure to demonstrate its combination of large angular coverage and turn-on sensitivity—in both planar and bent conditions—and a harvesting ability up to a distance of 2.83 m in its current configuration and exceeding 180 m using state-of-the-art rectifiers enabling the harvesting of several μW of DC power (around 6 μW at 180 m with 75 dBm EIRP).

Similar content being viewed by others

Compact high-efficiency energy harvesting positive and negative DC supplies voltage for battery-less CMOS receiver

Marwa Mansour & Islam Mansour

Adhoc mobile power connectivity using a wireless power transmission grid

Pawan Gaire, Dieff Vital, … Shubhendu Bhardwaj

Body-coupled power transmission and energy harvesting

Jiamin Li, Yilong Dong, … Jerald Yoo

Introduction

Our era is witnessing a rapid development in the field of millimeter-wave (mm-wave) and Internet of Things (IoT) technologies with a projected 40 billion IoT devices to be installed by 2025 1 . This could result in a huge number of batteries needing to be continuously charged and replaced. The design and realization of energy-autonomous, self-powered systems: the perpetual IoT, is therefore highly desirable. One potential way of satisfying these goals is through electromagnetic energy harvesting. A powerful source for electromagnetic scavenging is mm-wave energy, present in the fifth-generation (5G) of mobile communications bands (above 24 GHz), where the limits of allowable transmitted Effective Isotropic Radiated Power (EIRP) by the Federal Communications Commission (FCC) regulations are pushed beyond (reaches 75 dBm) that of their lower-frequency counterparts. Following the path loss model defined by the 3rd Generation Partnership Project Technical Report 3GPP TR 38.901 (release 16) in outdoor Urban Macro Line of Sight conditions (UMa LOS), the power density expected to be received at 28 GHz for a transmitted power of 75 dBm EIRP is 28 μW cm −2 at a distance of 100 m away from the transmitter. This demonstrates the ability of 5G to create a usable network of wireless power. In addition to the advantage of high transmitted power available at 5G, moving to mm-wave bands allows the realization of modular antennas arrays instead of single elements, thereby allowing a fine scaling of their antenna aperture, which can more than compensate for the high path loss at these frequencies through the addition of extremely-large gains 2 . However, one limitation accompanies large gain antennas: their inability to provide a large angular coverage. As the relative orientations of the sources and harvesters are generally unknown, the use of large aperture mm-wave harvesters may seem limiting and impossible. Individual rectennas, constituted of small antenna elements, can realistically be DC combined. However, this approach does not increase the turn-on sensitivity (lowest turn-on power) of the overall rectenna system: RF combination is needed.

Beamforming networks (BFNs) are used to effectively create simultaneous beam angular coverage with large-gain arrays, by mapping a set of directions to a set of feeding ports. An important class of these multiple networks is the microwave passive BFN that has been widely used in switched-beam antenna systems and applications. Hybrid combination techniques, based on Butler matrix networks, have been used in previous works for energy harvesting at lower frequencies 3 , 4 ,—more specifically at 2.45 GHz—to achieve wider angular coverage harvesting. However, these Ultra-High Frequency (UHF) arrays are impractically large for IoT applications and the implementation of their Butler matrices at higher frequencies would necessitate costly high-resolution fabrication. While sub-optimal—because of its large size—in the UHF band, the Rotman lens becomes the BFN of choice in the realm of mm-wave energy harvesting. Compared to their lower frequencies counterpart, fewer implementations are presented in the literature targeting energy harvesting at higher frequencies, more specifically 24 GHz and above. However, these systems later displayed in the table of comparison 5 , 6 , 7 , suffer from a narrow angular coverage.

In this paper, the authors demonstrate a full implementation of an entirely flexible, bending-resilient and simultaneously high gain and large angular coverage system for 5G/mm-wave energy harvesting based on a Rotman lens. For IoT applications, there is a benefit to making extremely low-profile devices that can conformally fit onto any surface in the environment such as walls, bodies, vehicles, etc. Therefore, thanks to the use of mm-waves, antennas with such features can be readily designed and fabricated. A Rotman lens-based rectenna has been first proposed in 8 , where a preliminary prototype and approach were presented, resulting in a quasi-flexible system, 80° angular coverage and 21-fold increase in the harvested power compared to a non-Rotman-based system. Here, the previously-predicted potential of 5G-powered nodes for the IoT and long-range passive mm-wave Radio Frequency IDentification (RFID) devices, is further taken advantage of, and effectively demonstrated. In order to do so, a thorough analysis of the lens itself—a structure that was not revealed in 8 —is first presented, exposing its key design parameters and resulting measured broadband behavior tested in both planar and bent conditions over more than 20 GHz of bandwidth. In addition, a scalability study of the approach, outlining the optimal size of such a system is reported, thereby proving the extent of the capability of providing a combination of good array factor and wide beam coverage. The novelty of this system also lies in the realization of a fully-flexible 28 GHz Rotman-lens-based rectenna system, completed by the design of a new DC combiner on a flexible 125 μm-thin polyimide Kapton substrate. The new DC combiner uses a reduced number of bypass diodes and increases the angular coverage of the system by more than 30% compared to 8 . Furthermore, the frequency-broadband behavior enabled by the use of the Rotman lens makes the full rectenna system bending-resilient, a property now demonstrated through its characterizations in flexing and conformally-mounted configurations. Finally, the system’s potential for long-range mm-wave harvesting is expressed for the first time, by reporting an unprecedented harvesting range of 2.83 m.

Experiments, results and discussions

Rotman lens scalability study for harvesting applications.

The Rotman lens, introduced in the 1960s, constitutes one of the most common and cost-effective designs for BFNs and is commonly utilized to enable multibeam phased array system 9 and wide-band operation, thanks to its implementation of true-time-delays 10 . By properly tuning the shape of the lens according to the geometrical optics approximation with the goal of focalizing plane waves impinging on the antenna side of the lens to different focal points on the beam-ports side of the lens, one achieves a lens-shaped structure with two angles of curvatures: one on the beam-ports side, and the other on the antenna side 11 . Because the lens is capable of focusing the energy coming from a given direction into its geometrically-associated beam port, the proposed scheme loads each of these ports with a rectifier, thereby channeling the energy coming from any direction to one of the rectifiers as shown in Fig. 1 a. This subsection investigates the effect of varying the number of antenna ports Na and beam ports Nb in the Rotman lens on its maximum array factor and angular coverage. The ( Na , Nb ) set, resulting in the best combination, will define the Rotman lens design parameters used for this work. Structures of varying sizes were designed using Antenna Magus and identical material parameters (substrate, conductors) as the ones of the presented design, before being simulated in CST STUDIO SUITE 2018. The simulated data was then processed in MATLAB to output the array factors created by the respective lens structures using a modified version of Eq. ( 1 ) 12 , presented next in Eq. ( 2 ):

where AF , n , Na , k , d , \(\theta\) and \(\beta\) are, respectively, the lossless array factor, the antenna number, the total number of antenna ports, the wave vector, the spacing between the elements, the direction of radiation and the difference in phase excitation between the elements. Since this formula describes a lossless array with a single antenna port, we introduced the following equation that takes into account the losses induced by the feeding network as well as the introduction of multiple feeding ports.

where \(AF_j\) and \(S_{nj}\) are, respectively, the array factor for beam port j and the S parameters between antenna ports n and beam ports j . The maximum value of the array factors as well as their total (accounting for the aggregated coverage of all ports) 3 dB beamwidths where then tabulated. The five simulated lenses had the following ( Na , Nb ) combinations: (4,3), (8,6) representing the system implemented in this work, (16,12), (32,24) and (64,48). Figure 1 b shows the increase in the array factor until reaching a peak of around 7.8 dB for a lens surrounded by 16 antennas and 12 beam ports, after which the array factor starts dropping, down to approximately 5.2 dB for a 64 antennas structure with 48 beam ports. The array factor reduction is explained by the increased losses within the lens accompanied by the increase of complexity and internal reflections, as the lens grows in electrical size. The same plot shows the decrease in angular coverage from 180° with 4 antennas down to 80° with 64 antennas. This study shows that the combination composed of eight antennas and six beam ports, offers a nearly optimal compromise, with these materials, between a high array factor of 5.95 dB and a 120° total angular coverage, while maintaining a reasonable number of antennas and beam ports. It should be noted that the choice of the number of beam ports is related to the 3dB-beamwidth of the individual antennas, the reason for which will be detailed later.

( a ) Dual combining (RF + DC) enabled by the use of the Rotman lens between the antennas and the rectifiers, ( b ) plot of the simulated maximum array factors and angular coverages for different-size Rotman lenses and ( c ) picture of the fabricated Rotman lens structure.

Flexible broadband Rotman lens design

After setting the number of antenna ports and beam ports, the design was printed on flexible copper-clad Liquid Crystal Polymer (LCP) substrate ( \(\varepsilon _r = 3.02\) and \(\hbox {h}= 180\,\upmu \hbox {m}\) ) using an inkjet-printed masking technique followed by etching, resulting in the structure shown in Fig. 1 c. It should be noted that the use of impedance-matched dummy ports is common with Rotman lenses 13 , 14 , 15 , 16 . Nevertheless, the goal in the implementation hereby described is not (as is usually the case) the generation of clean beam patterns with low side-lobe levels. Here, the lens’ properties are used for harvesting. Consequently, as long as the presence of the side lobes does not significantly interfere with the level of the array factor at broadside, side lobes are of no concern. Such a structure, including eight antenna ports and six beam ports—and, therefore, six radiating directions—was designed, simulated, and tuned. The structure, shown in Fig. 1 c, with the antenna ports connected to matched loads, was then tested in planar and bent configurations—cylinders with different bending radii ranging from 1.5 to 2.5 in. radii—to assess the effect of bending on the S parameters behavior. Figure 2 a shows the measured reflection coefficient of the Rotman lens at beam port 4 for four different scenarios, in comparison with the simulated structure in a planar position. The results reveal the Rotman lens’ ability to be mounted on curved surfaces down to a radius R = 1.5″, while maintaining a stable matching and minuscule losses compared to being held in a planar position.

( a ) Plot of the simulated and measured reflection coefficients at beam port 4 under planar and bent conditions and ( b ) Plots of the maximum array factors and angular directions of beam ports P1, P3 and P5 with respect to frequency.

The gain and angular bandwidths of this structure—defined by the frequency range in which the maximum array factor and angular direction per beam are stable within 3 dB and 5° respectively,—are studied next. The ultimate assessment of these properties involves calculating the beams’ magnitude and angular directions over a wide range of frequencies 17 , in order to ascertain their stability or lack thereof. For this purpose, the maximum array factors were calculated and the beams’ angular directions were extracted and plotted in Fig.  2 b for the first, third and fifth beam ports, P1, P3 and P5, representing the edge, secondary and central beams in this symmetrical structure. These plots prove the unique capabilities offered by the Rotman lens; although the Rotman lens is designed at a specific frequency—28 GHz in this work—this analysis proves that both the magnitude and the angular direction of the beams remain relatively stable over a very wide frequency range. In Fig. 2 b, three plots refer to the maximum array factors of the three beam ports, where minor fluctuations between 4 and 7 dB are observed over the range from 10 to 43 GHz for ports P3 and P5 and similar fluctuations over a fairly reduced frequency range for the extreme edge beam P1. On the same graph, three plots present the angular direction’s stability of P1, P3 and P5 beams, where P3 (in particular) preserves its angular direction over 33 GHz of bandwidth. The lens’ angular coverage resides between ports 1 and 6 and can be extracted from Fig. 2 b. Knowing that the structure is symmetrical and that beam port P1 is at around \({-54}^\circ\) , the overall structure covers an angle larger than 100° in front of the lens, a result further detailed in the next subsection. It should be noted that such a beamwidth is maintained over a large angular bandwidth exceeding 20 GHz, as shown in Fig. 2 b. This study demonstrates the stability and robustness of a low-cost, printed and flexible mm-wave Rotman lens structure, tested with respect to bending and frequency, and supports the choice of such an architecture at the heart of the harvesting system proposed in this work.

Flexible, high-gain and wide-angular-coverage mm-wave Rotman-lens-based antenna array

Eight of the linear antenna sub-arrays introduced in 8 were then added to the antenna ports of the array, and its beam-ports were extended by microstrip lines to enable their connection to end-launch \({2.92}\,\upmu \hbox {m}\) connectors. The antenna sub-array consists of five serially-fed patch antenna elements, providing an operation centered at 28.55 GHz with a reflection coefficient \(S_{11}\) lower than \({-20}\)  dB within this range. Their E-plane beamwidth of about \({18}^\circ\) (provided by the five antennas) is appropriate for most use cases, where environments expand mostly horizontally. Its simulations showed a gain of 13 dBi and a H-plane beamwidth of 80° in the plane perpendicular to the linear array. In this configuration, six beams were chosen to intersect at angles providing 3dB lower gain than broadside. Eight antennas provide a 3dB-beamwidth of 15°, which covers a total of \(6\times {18}^\circ = {108}^\circ\) in front of the array. The design was then also printed on flexible LCP substrate, resulting in the structure shown in Fig. 3 a, mounted on a 1.5″ radius cylinder. The radiation properties of the lens-based antenna system were simulated using the time-domain solver of CST STUDIO SUITE 2018, resulting in the six gain plots shown in Fig. 3 b. The gain of the Rotman lens at every port was also accurately measured using a 20 dBi transmitter horn antenna and by terminating all five remaining ports with a \({50}\,\Omega\) load for every port measurement to guarantee the proper operation of the lens. Both simulated and measured radiation patterns (shown in Fig. 3 b) display a remarkable similarity with a measured gain of approximately 17 dBi, and an angular coverage of around 110°, thereby validating the operation of the antenna array. The gains on the first three ports were also measured for the bent structure over a curvature of 1.5″ radius, shown in Fig. 3 a and compared to the measured results on a planar surface. The previous subsection in addition to previous works 18 , 19 have demonstrated that the performance of the Rotman lens is not deteriorated by wrapping or folding the structure compared to its conventional planar counterpart. However, after adding the antenna arrays, bending the structure can indeed have effects on its phase response, especially if the structure is large and the bending is severe. Figure  3 c shows the gains of P1, P2 and P3 for the two scenarios (three ports only because the structure is symmetrical), demonstrating again the ability of the lens in maintaining a stable gain (especially over the center beams) upon bending. The beam located at the edge, however, suffers additional deterioration in received power under bending, because of the shift of the source away from the broadside of the bent antenna arrays.

( a ) Picture of the flexible Rotman-lens-based antenna array, ( b ) measured (solid lines) and simulated (dashed lines) gains of the antenna array held in a planar position and ( c ) measured gains of the antenna array for beams P1, P2 and P3 only (because of the symmetry of the structure) in planar and bent conditions.

Fully-flexible 28 GHz Rotman lens-based system

Rotman-lens-based rectenna.

In this section, the fully-flexible rectenna system—based on the Rotman lens and a new DC combiner network—is presented. This architecture, shown in Fig. 4 a, consists of a series of eight antenna sub-arrays attached to the Rotman lens from one side, facing six rectifiers at the opposite side where DC serial combination is implemented. The basic rectenna elements, that are the antenna and the rectifier, are presented in details in 8 . The diode used in this work is the MA4E2038 Schottky barrier diode from Macom. The Rotman-based rectenna was first characterized as a function of its received power density. The system was positioned at a specific harvesting angle (approximately \(-25^\circ\) ) and illuminated with a horn antenna with a gain of 20 dBi, placed at a distance of 52 cm away from the rectenna array, within the far field region starting at 23 cm, and outputting powers ranging from 18 to 25 dBm, corresponding to an RF input power sweep from around − 9 dBm to − 2 dBm. The array was loaded with its optimal load impedance of 1  \(\hbox {k}\Omega\) , corresponding to the optimal load of a single rectifier—since only one rectifier will be “ON” at a time, given that the Rotman lens focalizes all the power to one beam port depending on the direction of the incoming wave—as detailed earlier. The results of this experiment are shown in Fig. 4 b, where the harvested voltages and powers of the array are shown. It can be observed that, at low powers, the Rotman-based rectenna effortlessly produces an output. The Rotman-based rectenna turns on well below − 6 dBm cm −2 , which compares quite favorably to the literature 6 . The output voltage of the rectenna was also measured over its operating frequency range. Like in the first experiment, the system was positioned at the same harvesting angle, at a range of 25 cm away from the source’s horn antenna. The output voltages under open load conditions were recorded and plotted, as shown in Fig. 4 c for the Rotman lens-based rectenna, for \(P_d = {9}\,{\hbox{dBm cm}}^{-2}\) , \(P_d = {10.5}\,{\hbox{dBm cm}}^{-2}\) and \(P_d = {12}\,{\hbox{dBm cm}}^{-2}\) incident power densities. The plots present a wide frequency coverage—from 27.8 to 29.6 GHz.

( a ) Picture of the fully-flexible Rotman-based rectenna, ( b ) plot of the measured voltages and output powers versus incident power density for the Rotman-based rectenna and ( c ) plot of the measured voltages with respect to frequency for the Rotman-based rectenna.

Flexible DC combining network

Power summation is very critical when it comes to the unbalanced rectification outputs produced from realistic RF sources, and can be implemented differently depending on its costs and benefits 20 .

This paper does not rely on a direct voltage summation topology (i.e. back-to-back RF diodes); however, it introduces a minimalist architecture relying on a total of \(2\times (N-1)\) bypass diodes, where N is the number of RF or rectifying diodes. Equipped with a low turn-on voltage of 0.1 V, the Toshiba 1SS384TE85LF bypass diodes used in the DC combiner design create a low resistance current path around all other rectifiers that received very low or close to zero RF power. This topology is optimal when only one diode is turned on, which can be assumed if a single, dominant source of power irradiates this particular design from a given direction. This new combiner circuit is shown in the schematic of Fig. 5 a. This simplified schematic—shown for four rectifying diodes—uses different colors to highlight the paths that the current will take for every case where an RF diode turning “ON” while the serially-connected diodes are “OFF”. This DC combiner was then fabricated on a flexible \({125\,\upmu \mathrm{m}}\) -thin polyimide Kapton substrate and connected to the Rotman lens-based rectenna through a series of single connectors to make the entire system fully flexible and bendable. The harvested power under a load of 1  \(\hbox {k}\, \Omega\) versus the angle of incidence of the mm-wave energy source for the Rotman-lens-based rectenna is compared for both rigid (presented in 8 , and relying on \(2\times N\) bypass diodes) and flexible new DC combiners. For this experiment, a horn transmitter antenna was used to send 25 dBm of RF power at 28.5 GHz to the lens placed 70 cm away, as shown in Fig. 5 b, while the array was precisely rotated in angular increments of 5°. Figure 6 a shows that the new DC combiner, with a reduced number of diodes, was able to provide a complete angular coverage of almost 110° over the entire lens spectrum as presented in Fig. 3 b, thus solving the voltage nulling occurring at the first and last ports, using the rigid DC combiner adopted previously in 8 . The new DC combiner offers therefore, an increase of more than 30% in the system’s spatial angular in addition to enabling a fully-bendable structure due to the unique fabrication on flexible Kapton substrate and connection to the rectenna using individual interconnects.

( a ) Rotman-based rectenna power summation network and ( b ) picture of the setup used to measure the angular response of the rectenna.

( a ) Plot of the measured harvested powers by the rectenna with respect to the source’s incidence angle for the two DC combiners, rigid and flexible and ( b ) plots of the measured harvested powers and voltages with respect to the incident power density under different load conditions for the Rotman lens rectenna with and without the flexible DC combiner.

As mentioned earlier, the DC combiner is mainly used with the Rotman-lens-based rectenna to automatically direct the active rectifier’s output to a single DC common port, independent of which port this might be. An alternative to the DC combiner in the Rotman lens-based system, would be to manually connect to the active port if the location of the source were known. To study the effect of the implemented DC combiner on the turn-on sensitivity of the system, the output voltage of the rectenna was measured for a specific source location with and without the combiner over a range of RF transmitted power and load variations; the direction was chosen such that the non-DC-combined rectifier would output its maximum power. Figure  6 b shows eight different plots where three of them represent the harvested power with a direct connection to the active rectifier for 1  \(\hbox {k}\Omega\) , 10  \(\hbox {k}\Omega\) and 100  \(\hbox {k}\Omega\) conditions. Plotted with the same colors are the other three, representing the harvested power with the addition of the DC combiner for the same load values. The last two plots display the measured voltages with and without the combiner under open load conditions. The rectenna was placed 61 cm away from the transmitter horn antenna and the power was swept from 10 to 25 dBm. The results show the performance superiority in all considered load conditions when the contact is made directly to the rectifier and not through the DC combiner. The lens-based system is able to achieve a turn-on power as low as \(-15\,{\hbox{dBm cm}}^{-2}\) in this case. This behavior is explained by the voltage drop introduced by the bypass diodes present in the combiner—that consistently decrease the expected output voltage by 0.1 to 0.2 V—when one or two diodes are, respectively, added to the current path. The variation of load values also shows that the rectenna can achieve better efficiencies at lower loads. More importantly, the reduction in the turn-on sensitivity—the minimum power density required output 10 mV—induced by the combiner is only of about 2 dB in loaded conditions, while the combiner enables an increase in the angular coverage of the rectenna system from about 18° to 110°. The remarkable angular and high-power turn-on sensitivity offered by the Rotman-lens-based rectenna are finally benchmarked using the following table for comparison with several state-of-the-art works, as presented in literature. In Table  1 , the striking performance of the proposed system is displayed, highlighted by its flexibility and ability of achieving an angular coverage as large as 110° at extremely high turn-on sensitivity, thereby allowing mm-wave long-range harvesting in ad-hoc and conformal-mounting implementations.

Rectenna system performance under bending

This section displays the operation of the Rotman-lens-based system under different bending scenarios. This and previous work 18 , 19 show that the lens is able to maintain an efficient electromagnetic energy distribution across the output ports under convex and concave flexing conditions. The lens-based rectenna was placed on cylinders with different curvatures, 70 cm away from the transmitter sending 25 dBm of power at 28.5 GHz, as shown on Fig. 7 a. The voltage was collected using a load of 1  \(\hbox {k}\Omega\) for the planar and three bent conditions and plotted in Fig.  7 b with respect to the source’s angle of incidence. The graph shows an unprecedented consistency and stability in the system’s scavenging and rectification abilities, knowing that several sub-systems are exposed to warping and the pressures of bending: the antenna sub-arrays, the Rotman lens and the rectifiers. Slight attenuation can be observed at the edges, but the system otherwise performs unimpeded by the bending. This remarkable property qualifies this system as a perfect candidate for use in wearables, smart phones and ubiquitous, conformal 5G energy harvesters for IoT nodes.

( a ) Picture of the flexible Rotman lens-based rectenna placed on a 1.5″ radius cylinder and ( b ) measured harvested powers versus incidence angles for different curvatures, ( c ) long-range harvesting testing setup.

Long-range harvesting

As described earlier, one of the main appeals of the proposed approach is its ability to use the high EIRPs allowed for 5G base-stations while guaranteeing an extended beam angular coverage, which is a necessary feature for ad-hoc ubiquitous harvesting implementations. In order to demonstrate the lens based-rectenna for longer-distance harvesting and detect that maximum range, a high-performance antenna system—comprised of a 19 dBi conical horn antenna and a 300 mm-diameter PTFE dielectric lens (for high directivity) providing an additional 10 dB of gain—was used as shown in Fig. 7 c. With a transmitted power of 25 dBm (and an associated EIRP of approximately 54 dBm), corresponding to an incident power density of approximately − 6 dBm cm −2 , the lens-based rectenna displayed an extended range of 2.83 m under open load conditions, with an output voltage around 10 mV, thereby demonstrating (to our knowledge) the longest-ranging rectenna demonstration at mm-wave frequencies. With a transmitter emitting the allowable 75 dBm EIRP, the theoretical maximum reading range of this rectenna could extend to 16 m. In addition, the use of advanced diodes—designed for applications within the 5G bands and enabling rectifiers’ sensitivities similar to that common at lower (UHF) frequencies—are showing a potential path towards achieving a turn-on sensitivity of the rectifiers as low as − 30 dBm 21 , 22 . If this were practically applied to the Rotman lens system presented in this work, the harvesting range could be extended beyond 180 m (where the received power density for a transmitted power of 75 dBm is \({7.8}\,\upmu \hbox {W cm}^{-2}\) ), which is only slightly smaller than the recommended cell size of 5G networks 23 . This observation enables the striking idea that future 5G networks could be used not only for tremendously-rapid communications, but also as a ubiquitous wireless power grid for IoT devices.

Through the use of the Rotman lens, this paper demonstrates that the usual paradigm constrained by the (often considered fundamental) trade-off between the angular coverage and the turn-on sensitivity of a wireless harvesting system can be broken. Using the reported architecture, one can design and fabricate flexible mm-wave harvesters that can cover wide areas of space while being electrically large and benefit from the associated improvements in link budget (from source to harvester) and, more importantly, turn-on sensitivity. The approach has been shown, however, to only be scalable up to the degree where the additional incremental losses introduced by the growing lens counterbalance the increase in the aperture of the rectenna. Nevertheless, this inflection point only appears (in the particular context considered in this paper) after the arraying of 16 elements, or up to a scale of \(8\lambda\) . In the 5G Frequency Range 2 (FR2), this translates to harvesters of 4.5 cm to 9.6 cm in size, which are perfectly suited for wearable and ubiquitous IoT implementations. With the advent of 5G networks and their associated high allowed EIRPs and the availability of diodes with high turn-on sensitivities at 5G frequencies, several \({\upmu \hbox {W}}\) of DC power (around 6  \({\upmu \hbox {W}}\) with 75 dBm EIRP) can be harvested at 180 m. Such properties may trigger the emergence of 5G-powered nodes for the IoT and, combined with the long-range capabilities of mm-wave ultra-low-power backscatterers 24 , of long-range passive mm-wave RFIDs.

Mercer, D. Global Connected and IoT Device Forecast Update . https://www.strategyanalytics.com/access-services/devices/connected-home/consumer-electronics/reports/report-detail/global-connected-and-iot-device-forecast-update (2019).

Hester, J. G. D. & Tentzeris, M. M. Inkjet-printed flexible mm-wave Van-Atta reflectarrays: A solution for ultra-long-range dense multi-tag and multi-sensing chipless RFID implementations for IoT smart skins. IEEE Trans. Microw. Theory Tech. 57 , 1303–1309 (2017).

Google Scholar  

Lee, D.-J., Lee, S.-J., Hwang, I.-J., Lee, W.-S. & Yu, J.-W. Hybrid power combining rectenna array for wide incident angle coverage in RF energy transfer. IEEE Trans. Microw. Theory Tech. 65 , 3409–3418 (2017).

Article   ADS   Google Scholar  

Kuek, J. et al. A compact butler matrix for wireless power transfer to aid electromagnetic energy harvesting for sensors. In 2017 IEEE Asia Pacific Microwave Conference (APMC) 334–336 (2017).

Bito, J. et al. Millimeter-wave ink-jet printed RF energy harvester for next generation flexible electronics. In 2017 IEEE Wireless Power Transfer Conference (WPTC) 1–4 (2017).

Ladan, S., Guntupalli, A. B. & Wu, K. A high-efficiency 24 GHz rectenna development towards millimeter-wave energy harvesting and wireless power transmission. IEEE Trans. Circuits Syst. I Regul. Pap. 61 , 3358–3366 (2014).

Article   Google Scholar  

Okba, A., Takacs, A., Aubert, H., Charlot, S. & Calmon, P.-F. Multiband rectenna for microwave applications. C. R. Phys. 18 , 107–117 (2017).

Article   ADS   CAS   Google Scholar  

Eid, A., Hester, J. & Tentzeris, M. M. A scalable high-gain and large-beamwidth mm-wave harvesting approach for 5g-powered IoT. In 2019 IEEE MTT-S International Microwave Symposium (IMS) 1309–1312 (2019).

Wang, K., Gu, J.-F., Ren, F. & Wu, K. A multitarget active backscattering 2-d positioning system with superresolution time series post-processing technique. IEEE Trans. Microw. Theory Tech. 65 , 1751–1766 (2017).

Rotman, R., Tur, M. & Yaron, L. True time delay in phased arrays. Proc. IEEE 104 , 504–518 (2016).

Rotman, W. & Turner, R. Wide-angle microwave lens for line source applications. IEEE Trans. Antennas Propag. 11 , 623–632 (1963).

Balanis, C. A. Antenna Theory: Analysis and Design (Wiley, New York, 2016).

Attaran, A., Rashidzadeh, R. & Kouki, A. 60 GHz low phase error rotman lens combined with wideband microstrip antenna array using LTCC technology. IEEE Trans. Antennas Propag. 64 , 5172–5180 (2016).

Hassanien, M. A., Hahnel, R. & Plettemeier, D. Wideband rotman lens beamforming technique for 5g wireless applications. In 2019 2nd International Conference on Computer Applications and Information Security (ICCAIS) 1–5 (2019).

Tekkouk, K., Ettorre, M. & Sauleau, R. SIW rotman lens antenna with ridged delay lines and reduced footprint. IEEE Trans. Microw. Theory Tech. 66 , 3136–3144 (2018).

Jastram, N. & Filipovic, D. S. Design of a wideband millimeter wave micromachined rotman lens. IEEE Trans. Antennas Propag. 63 , 2790–2796 (2015).

Article   ADS   MathSciNet   Google Scholar  

Darvazehban, A., Manoochehri, O., Salari, M. A., Dehkhoda, P. & Tavakoli, A. Ultra-wideband scanning antenna array with rotman lens. IEEE Trans. Microw. Theory Tech. 65 , 3435–3442 (2017).

Rahimian, A., Alfadhl, Y. & Alomainy, A. Design and performance of a flexible 60-GHz rotman lens-based array beamformer. In 12th European Conference on Antennas and Propagation (EuCAP 2018) 1–2 (2018).

Vo Dai, T. K. & Kilic, O. Compact rotman lens structure configurations to support millimeter wave devices. Prog. Electromag. Res. B 71 , 91–106 (2016).

Parks, A. N. & Smith, J. R. Active power summation for efficient multiband RF energy harvesting. In 2015 IEEE MTT-S International Microwave Symposium (IMS) 1–4 (2015).

Eid, A., Hester, J., Tehrani, B. & Tentzeris, M. Flexible w-band rectifiers for 5g-powered IoT autonomous modules. In 2019 IEEE International Symposium on Antennas and Propagation and USNC-URSI Radio Science Meeting 1163–1164 (2019).

Gao, H., Leenaerts, D. M. & Baltus, P. A 58-64 GHz transformer-based differential rectifier in 40 nm CMOS with-12 dBm sensitivity for 1 V at 64 GHz. In 2019 IEEE MTT-S International Microwave Symposium (IMS) 1306–1308 (2019).

Sulyman, A. I. et al. Radio propagation path loss models for 5g cellular networks in the 28 GHz and 38 GHz millimeter-wave bands. IEEE Commun. Mag. 52 , 78–86 (2014).

Hester, J. G. & Tentzeris, M. M. A mm-wave ultra-long-range energy-autonomous printed RFID-enabled van-Atta wireless sensor: At the crossroads of 5g and IoT. In 2017 IEEE MTT-S International Microwave Symposium (IMS) 1557–1560 (2017).

Download references

Acknowledgements

This work was supported by the Air Force Research Laboratory and the NSF-EFRI. The work was performed in part at the Georgia Tech Institute for Electronics and Nanotechnology, a member of the National Nanotechnology Coordinated Infrastructure (NNCI), which is supported by the National Science Foundation (Grant ECCS-1542174).

Author information

Authors and affiliations.

School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

Aline Eid, Jimmy G. D. Hester & Manos M. Tentzeris

Atheraxon, Atlanta, GA, 30308, USA

Jimmy G. D. Hester

You can also search for this author in PubMed   Google Scholar

Contributions

A.E. and J.H. conceived the idea, designed, and simulated the antenna arrays, rectifiers, Rotman lens, DC combiners and full rectennas. They also performed the measurements, interpreted results and wrote the paper. M.T. supervised the research and contributed to the general concept and interpretation of the results. All authors reviewed the manuscript.

Corresponding author

Correspondence to Aline Eid .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Eid, A., Hester, J.G.D. & Tentzeris, M.M. 5G as a wireless power grid. Sci Rep 11 , 636 (2021). https://doi.org/10.1038/s41598-020-79500-x

Download citation

Received : 18 August 2020

Accepted : 09 December 2020

Published : 12 January 2021

DOI : https://doi.org/10.1038/s41598-020-79500-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Wirelessly powered large-area electronics for the internet of things.

• Luis Portilla
• Kalaivanan Loganathan
• Vincenzo Pecunia

Nature Electronics (2022)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Wireless technologies for the connectivity of the future

Filipe d. cardoso.

1 ESTSetúbal, Polytechnic Institute of Setúbal and INESC-ID, Setúbal, Portugal

Vlatko Lipovac

2 University of Dubrovnik, Dubrovnik, Croatia

Luis M. Correia

3 IST/INESC-ID, University of Lisbon, Lisbon, Portugal

This Special Issue originates from the international conference EuCNC 2020 (European Conference on Networks and Communications), which was planned to be held in June 2020 in Dubrovnik (Croatia), but due to the COVID-19 pandemic was changed to an Online Conference. The Technical Programme Chairs of the conference have selected the best papers and invited authors to submit an extended version of their paper, by at least one third of their length. Only the top ranked papers were invited to this Special Issue, in order to fulfil its purpose. The main target was to collect and present quality research contributions in the most recent activities related to systems and networks beyond 5G, already presenting ideas for 6G. Through this Special Issue, the state-of-the-art is presented and the new challenges are highlighted, regarding the latest advances on systems and network perspectives that are already being positioned beyond 5G, bridging as well with the evolution of 5G, including applications and trials. Therefore, the motivation for this Special Issue is to present the latest and finest results on the evolution of research of mobile and wireless communications, coming, but not exclusively (since EuCNC is a conference open to the whole research community), from projects co-financed by the European Commission within its R&D programmes.

5G system aspects have already entered the commercial phase, and its networks ones (in the so-called Stand-Alone approach) will soon reach this phase. This means that, from the research viewpoint, work has started to address what will not be implemented in 5G, not only extending concepts for further phases of 5G but also initiating the views on what 6G may be composed of. From the viewpoint of backward compatibility, it is important that 6G potential technologies may bridge with 5G ones, but it is also necessary that new concepts are put forward. In fact, there are already appearing coordinated activities and projects in some countries addressing 6G, which shows the importance of this topic. Some current topics remain important, e.g., new frequency bands (extending beyond mm waves), mobile IoT, softwarisation of networks, network architectures, wearables and bridging with body area networks, security and privacy, and artificial intelligence in communications, among others, but one can expect that new ones are proposed. The scope of this Special Issue is to focus on new approaches beyond 5G, but still bridging with the foreseeable implementation of 5G.

In what follows, one presents a brief overview of the papers composing the current Special Issue.

End-to-End Network Slice Architecture and Distribution Across 5G Micro-Operator Leveraging Multi-Domain and Multi-Tenancy , authored by Idris Badmus, Abdelquoddouss Laghrissi, Marja Matinmikko-Blue and Ari Pouttu, addresses a sophisticated end-to-end network slicing architecture for different deployment scenarios of the local 5G micro-operator concept. Local 5G networks are emerging as a new form for 5G deployment, targeting service delivery for vertical-specific purposes and other local users. These networks are also known as micro-operator networks for which prior work has established different deployment scenarios, namely Closed, Open and Mixed Networks. To achieve network flexibility, customization and privacy required by various vertical sectors, such as industry, health and energy, it is essential to have a well-defined network slicing architecture and adequate implementation procedure. In this paper, a sophisticated end-to-end network slicing architecture is proposed for different deployment scenarios of the local 5G micro-operator concept. The proposed architecture incorporates a broad four-layer concept, leveraging a multi-tenancy layer for different tenants and their end users, a descriptive service layer, a multi-domain slicing Management and Orchestration (MANO) layer, and a resource layer. The authors further propose a network slice instance (NSI) communication service distribution technique for local 5G micro-operators. This is achieved by expanding/leveraging the communication service management function (CSMF) in the multi-tenant layer into a multi-tenant manager and an orchestrator of communication services. In addition, it is described how the communication service orchestrator will address all the possible multitenant-slice situations during the distribution of a network slice instance to multiple tenants. The novel methods described in the paper present a solution for not only network slice communication service distribution across different micro-operator’s tenants but also for future use cases, especially, when the allocated slice is responsible for multiple tenants or when a tenant request multiple NSIs.

Actor-Critic Learning based Energy Optimization for UAV Access-and-Backhaul Networks , authored by Yaxiong Yuan, Lei Lei, Thang X. Vu, Symeon Chatzinotas, Sumei Sun and Björn Ottersten, addresses energy minimization with a limited power supply for both backhaul and access links in unmanned aerial vehicle (UAV)-assisted networks. The UAV acts as an aerial base station which acquires the requested data via backhaul link and then serves ground users (GUs) through an access network. In this paper, the authors investigate an energy minimization problem with a limited power supply for both backhaul and access links. The difficulties for solving such a non-convex and combinatorial problem lie at the high computational complexity/time. In solution development, the approaches from both actor-critic deep reinforcement learning (AC-DRL) and optimization perspectives are considered. Firstly, two offline non-learning algorithms, i.e., an optimal and a heuristic algorithms, based on piecewise linear approximation and relaxation are developed as benchmarks. Secondly, towards real-time decision making, the conventional AC-DRL are improved and two learning schemes: AC-based user group scheduling and backhaul power allocation (ACGP), and joint AC-based user group scheduling and optimization-based backhaul power allocation (ACGOP) are proposed. Numerical results show that the computation time of both ACGP and ACGOP are reduced tenfold to 100-fold compared to the offline approaches, and ACGOP is better than ACGP in energy savings. The results also verify the superiority of proposed learning solutions in terms of guaranteeing the feasibility and minimizing the system energy compared to the conventional AC-DRL.

An Experimental Publish-Subscribe Monitoring Assessment to Beyond 5G Networks , authored by Ramon Perez, Jaime Garcia-Reinoso, Aitor Zabala, Pablo Serrano and Albert Banchs, is focused on fifth-generation (5G) of mobile networks designed to accommodate different types of use cases, each of them with different and stringent requirements and key performance indicators (KPIs). To support the optimization of the network performance and validation of the KPIs, there exists the necessity of a flexible and efficient monitoring system, capable of realizing multi-site and multi-stakeholder scenarios. Nevertheless, for the evolution from 5 to 6G, the network is envisioned as a user-driven, distributed Cloud computing system where the resource pool is foreseen to integrate the participating users. In this paper, the authors present a distributed monitoring architecture for Beyond 5G multi-site platforms, where different stakeholders share the resource pool in a distributed environment. Taking advantage of the usage of publish-subscribe mechanisms adapted to the Edge, the developed lightweight monitoring solution can manage large amounts of real-time traffic generated by the applications located in the resource pool. The performance of the implemented paradigm was accessed, revealing some interesting insights about the platform, such as the effect caused by the throughput of monitoring data in performance parameters such as the latency and packet loss, or the presence of a saturation effect due to software limitations that impacts in the performance of the system under specific conditions. In the end, the performance evaluation process has confirmed that the Monitoring platform suits the requirements of the proposed scenarios, being capable of handling similar workloads in real 5G and Beyond 5G scenarios, then discussing how the architecture could be mapped to these real scenarios.

Synchronization in 5G Networks: a Hybrid Bayesian Approach towards Clock Offset/Skew Estimation and Its Impact on Localization , authored by Meysam Goodarzi, Darko Cvetkovski1, Nebojsa Maletic, Jesús Gutiérrez and Eckhard Grass, addresses clock synchronization in 5G networks, a major challenge when designing wireless networks. This paper focuses on tackling the time synchronization problem in 5G networks by adopting a hybrid Bayesian approach for clock offset and skew estimation. Furthermore, we provide an in-depth analysis of the impact of the proposed approach on a synchronization-sensitive service, i.e., localization. Specifically, the authors expose the substantial benefit of Belief Propagation (BP) running on Factor Graphs (FGs) in achieving precise network-wide synchronization. Moreover, the proposed approach takes advantage of Bayesian Recursive Filtering (BRF) to mitigate the time-stamping error in pairwise synchronization. Finally, the merit of hybrid synchronization by dividing a large-scale network into local synchronization domains and applying the most suitable synchronization algorithm (BP- or BRF-based) on each domain is revealed. The performance of the hybrid approach is then evaluated in terms of the Root Mean Square Errors (RMSEs) of the clock offset, clock skew, and the position estimation. According to the simulations, despite the simplifications in the hybrid approach, RMSEs of clock offset, clock skew, and position estimation remain below 10 ns, 1 ppm, and 1.5 m, respectively.

Outage Prediction for Ultra-Reliable Low-Latency Communications in Fast Fading Channels , authored by Andreas Traßl, Eva Schmitt Tom Hößler, Lucas Scheuvens, Norman Franchi, Nick Schwarzenberg and Gerhard Fettweis, deals with outage prediction approaches for Rayleigh and Rician fading channels. The addition of redundancy is a promising solution to achieve a certain Quality of Service (QoS) for ultra-reliable low-latency communications (URLLC) in challenging fast fading scenarios. However, adding more and more redundancy to the transmission results in severely increased radio resource consumption. Monitoring and prediction of fast fading channels can serve as the foundation of advanced scheduling. By choosing suitable resources for transmission, the resource consumption is reduced while maintaining the QoS. In this article, the authors present outage prediction approaches for Rayleigh and Rician fading channels. Appropriate performance metrics are introduced to show the suitability for URLLC radio resource scheduling. Outage prediction in the Rayleigh fading case can be achieved by adding a threshold comparison to state-of-the-art fading prediction approaches. A line of sight (LOS) component estimator is introduced that enables outage prediction in LOS scenarios. Extensive simulations have shown that under realistic conditions, effective outage probabilities of 10–5 can be achieved while reaching up-state prediction probabilities of more than 90%. We show that the predictor can be tuned to satisfy the desired trade-off between prediction reliability and utilizability of the link. This enables our predictor to be used in future scheduling strategies, which achieve the challenging QoS of URLLC with fewer required redundancy.

A Study About Signal Variation with Minor Receiver Displacement in a Meeting Room at 60 GHz: Measurements and Simulations , authored by Muhammad Usman Sheikh, Kalle Ruttik, Riku Jäntti and Jyri Hämäläinen, aims to study the impact of small receiver displacement on a signal propagation in a typical conference room environment at a millimeter wave (mmWave) frequency of 60 GHz. While channel measurements provide insights on the propagation phenomena, their use for the wireless system performance evaluation is challenging. Whereas, carefully executed three-dimensional ray tracing (RT) simulations represent a more flexible option. Nevertheless, a careful validation of simulation methodology is needed. The first target of this article is to highlight the benefits of an in-house built three-dimensional RT tool at 60 GHz, and shows the effectiveness of simulations in predicting different characteristics of the channel. To validate the simulation results against the measurements, two different transmitter (Tx) positions and antenna types along with ten receiver (Rx) positions are considered in a typical conference room. In first system configuration, an omnidirectional antenna is placed in the middle of the table, while in the second system configuration a directed horn antenna is located in the corner of the meeting room. After validating the simulation results with the measurement data, in the second part of this work, the impact of a small change i.e., 20 cm in the receiver position is studied. To characterize the impact the authors apply as performance indicators the received power level, root mean square delay spread (RMS-DS), and RMS angular spread (RMS-AS) in azimuth plane. The channel characteristics are considered with respect to the direct orientation (DO) i.e., the Rx antenna is directed towards the strongest incoming path. Different antenna configurations at the Tx and Rx side are applied to highlight the role of antenna properties on the considered channel characteristics. Especially, in the second system configuration the impact of different antenna half power beamwidth on different considered channel characteristics.

is highlighted through acquired simulation results. The validation of results shows the RMS error of only 2–3 dB between the measured and simulated received power levels for different Tx configurations in the direction of DO. Results indicate that only a small change of the Rx position may result a large difference in the received power level even in the presence of line-of-sight between the Tx and Rx. It is found that the STD of received power level across the room increases with the decrease in HPBW of the antenna. As can be expected, directed antennas offer lower value of RMS-DS and RMS-AS compared with isotropic antenna.

CARAMEL: Results on a Secure Architecture for Connected and Autonomous Vehicles Detecting GPS Spoofing Attacks , authored by Christian Vitale, Nikos Piperigkos, Christos Laoudias, Georgios Ellinas, Jordi Casademont et al., addresses the cybersecurity gaps introduced by the new technological domains adopted by modern vehicles applying, among others, advanced Artificial Intelligence and Machine Learning techniques. As a result, the H2020-CARAMEL enhances the protection against threats related to automated driving, smart charging of Electric Vehicles, and communication among vehicles or between vehicles and the roadside infrastructure. This paper focuses on the latter and presents the CARAMEL architecture aiming at assessing the integrity of the information transmitted by vehicles, as well as at improving the security and privacy of communication for connected and autonomous driving. The proposed architecture includes: (1) multi-radio access technology capabilities, with simultaneous 802.11p and LTE-Uu support, enabled by the connectivity infrastructure; (2) a MEC platform, where, among others, algorithms for detecting attacks are implemented; (3) an intelligent On-Board Unit with anti-hacking features inside the vehicle; (4) a Public Key Infrastructure that validates in real-time the integrity of vehicle’s data transmissions. As an indicative application, the interaction between the entities of the CARAMEL architecture is showcased in case of a GPS spoofing attack scenario. Adopted attack detection techniques exploit robust in-vehicle and cooperative approaches that do not rely on encrypted GPS signals, but only on measurements available in the CARAMEL architecture.

Cooperative Non-Orthogonal Multiple Access for Wireless Communication Networks by Exploiting the EXIT Chart Analysis , authored by Zeyad Elsaraf, Abbas Ahmed, Faheem Ahmad Khan, and Qasim Zeeshan Ahmed, addresses Successive Interference Cancellation (SIC) in Non-Orthogonal Multiple Access (NOMA). In the next generation of mobile communication networks, unprecedented challenges are required to be met, such as, much higher data rates and spectrum efficiency, lower latency, and massive connectivity. NOMA has recently been proposed as a promising technology to achieve much superior spectral efficiency compared to conventional orthogonal multiple access techniques employed in present communication systems. A salient feature of NOMA is its use of SIC to decode users’ information when multiple users are allowed to transmit in same time/frequency/code domain. In this paper, the authors aim to exploit an aspect of SIC, namely the availability of other users’ data to realize a cooperative NOMA system. EXtrinsic Information Transfer (EXIT) charts are utilized to examine the performance of proposed system in terms of user fairness while employing IRregular Convolutional Codes (IRCC)s. The EXIT chart using IRCC evaluates the convergence analysis for the proposed system. Further, to evaluate the system performances in cooperative NOMA system, the authors have derived the expressions for the achievable rates which are obtained independently and utilized them in evaluating the experimental data for the proposed NOMA model.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Filipe D. Cardoso, Email: [email protected] .

Vlatko Lipovac, Email: rh.udinu@cavopilv .

Luis M. Correia, Email: [email protected] .

This paper is in the following e-collection/theme issue:

Published on 10.4.2024 in Vol 26 (2024)

Methodological Frameworks and Dimensions to Be Considered in Digital Health Technology Assessment: Scoping Review and Thematic Analysis

Authors of this article:

• Joan Segur-Ferrer, BSS, PT, MSc   ; 
• Carolina Moltó-Puigmartí, BScPharm, PhD   ; 
• Roland Pastells-Peiró, BA, MA, MsC   ; 
• Rosa Maria Vivanco-Hidalgo, MD, MPH, PhD  

Agency for Health Quality and Assessment of Catalonia, Barcelona, Spain

Corresponding Author:

Joan Segur-Ferrer, BSS, PT, MSc

Agency for Health Quality and Assessment of Catalonia

Roc Boronat Street, 81-95, 2nd Fl

Barcelona, 08005

Phone: 34 935 513 900

Fax:34 935 517 510

Email: [email protected]

Background: Digital health technologies (dHTs) offer a unique opportunity to address some of the major challenges facing health care systems worldwide. However, the implementation of dHTs raises some concerns, such as the limited understanding of their real impact on health systems and people’s well-being or the potential risks derived from their use. In this context, health technology assessment (HTA) is 1 of the main tools that health systems can use to appraise evidence and determine the value of a given dHT. Nevertheless, due to the nature of dHTs, experts highlight the need to reconsider the frameworks used in traditional HTA.

Objective: This scoping review (ScR) aimed to identify the methodological frameworks used worldwide for digital health technology assessment (dHTA); determine what domains are being considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

Methods: The ScR was performed in accordance with the guidelines established in the PRISMA-ScR guidelines. We searched 7 databases for peer reviews and gray literature published between January 2011 and December 2021. The retrieved studies were screened using Rayyan in a single-blind manner by 2 independent authors, and data were extracted using ATLAS.ti software. The same software was used for thematic analysis.

Results: The systematic search retrieved 3061 studies (n=2238, 73.1%, unique), of which 26 (0.8%) studies were included. From these, we identified 102 methodological frameworks designed for dHTA. These frameworks revealed great heterogeneity between them due to their different structures, approaches, and items to be considered in dHTA. In addition, we identified different wording used to refer to similar concepts. Through thematic analysis, we reduced this heterogeneity. In the first phase of the analysis, 176 provisional codes related to different assessment items emerged. In the second phase, these codes were clustered into 86 descriptive themes, which, in turn, were grouped in the third phase into 61 analytical themes and organized through a vertical hierarchy of 3 levels: level 1 formed by 13 domains, level 2 formed by 38 dimensions, and level 3 formed by 11 subdimensions. From these 61 analytical themes, we developed a proposal for a methodological framework for dHTA.

Conclusions: There is a need to adapt the existing frameworks used for dHTA or create new ones to more comprehensively assess different kinds of dHTs. Through this ScR, we identified 26 studies including 102 methodological frameworks and tools for dHTA. The thematic analysis of those 26 studies led to the definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered in dHTA.

Introduction

Digital health technologies (dHTs) are driving the transformation of health care systems. They are changing the way in which health services are delivered, and showing great potential to address some of the major challenges that European health systems, including the Spanish National Health System (SNS), are facing, such as the progressive aging of the population [ 1 , 2 ]; the growing demand for health and long-term care services [ 2 ]; the rise in health care costs, increasing financial pressures on health and welfare systems [ 1 , 3 ]; and the unequal distribution of health services across different geographical regions [ 4 , 5 ]. In addition, dHT can improve the accessibility, sustainability, efficiency, and quality of health care systems [ 6 , 7 ], leading to their becoming a determinant of health on their own [ 6 , 8 ].

However, the digital transformation of health care systems and the implementation of dHT (eg, artificial intelligence [AI]–based solutions, data-driven health care services, or the internet of things) are slow and unequal across different European regions [ 9 , 10 ]. Some of the reasons for this are (1) the immaturity of regulatory frameworks for the use of dHTs [ 9 ], (2) the lack of funding and investment for the implementation of dHTs [ 9 ], (3) the lack of sufficient and appropriate infrastructures and common standards for data management [ 6 , 9 ], (4) the absence of skills and expertise of professionals and users [ 10 ], and (5) the scarcity of strong evidence regarding the real benefits and effects of dHTs on health systems and people’s well-being, as well as the cost-effectiveness of these technologies. This makes decision-making difficult, potentially leading to the development and reproduction of low-value and short-lived dHTs [ 6 , 11 ].

To overcome these challenges, harness the potential of dHTs, and avoid nonintended consequences, the World Health Organization (WHO) [ 4 , 11 ] states that dHTs should be developed under the principles of transparency, accessibility, scalability, privacy, security, and confidentiality. Their implementation should be led by robust strategies that bring together leadership, financial, organizational, human, and technological resources, and decisions should be guided by the best-available evidence [ 4 , 11 ].

Regarding this last aspect, health technology assessment (HTA), defined as a “multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its life cycle,” is a widely accepted tool to inform decision-making and promote equitable, efficient, and high-quality health systems [ 12 , 13 ].

Generally, HTA is conducted according to specific methodological frameworks, such as the HTA Core Model of the European Network for Health Technology Assessment (EUnetHTA) [ 14 ] and the guidelines for the development and adaptation of rapid HTA reports of the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS) [ 15 ]. These frameworks establish the methodologies to follow and the elements to evaluate. Although these frameworks are helpful instruments for evaluating various health technologies, they have certain limitations in comprehensively assessing dHTs. For this reason, in the past few years, different initiatives have emerged to adapt existing methodological frameworks or develop new ones. The objective is to consider additional domains (eg, interoperability, scalability) to cover the intrinsic characteristics of dHTs [ 16 - 18 ]. Examples of these initiatives are the Evidence Standard Framework (ESF) of National Institute for Health and Care Excellence (NICE) [ 19 ] or the Digi-HTA Framework of the Finnish Coordinating Center for Health Technology Assessment (FinCCHTA) [ 16 ]. Nonetheless, the majority of these frameworks have certain constraints, such as being designed for a particular socioeconomic or national setting, which restricts their transferability or suitability for use in other countries; the specificity or exclusion of certain dHTs, resulting in limitations in their application; or the limited evidence regarding their actual usefulness.

In this context, we performed a scoping review (ScR) with the aim of identifying the methodological frameworks that are used worldwide for the evaluation of dHTs; determining what dimensions and aspects are considered for each type of dHT; and generating, through a thematic analysis, a proposal for a methodological framework that is based on the most frequently described dimensions in the literature. This research focused mainly on mobile health (mHealth), non–face-to-face care models and medical devices that integrate AI, as these particular dHTs are the ones most frequently assessed by HTA agencies and units of RedETS.

Identifying Research Questions

This ScR followed by a thematic analysis answered the following research questions:

• What methodological frameworks currently exist for digital health technology assessment (dHTA)?
• What domains and dimensions are considered in dHTA?
• Do the different domains and dimensions considered depend on whether the dHT addressed is a non–face-to-face care model of health care provision, a mobile device (mHealth), or a device that incorporates AI?

Overview of Methods for Conducting the Scoping Review

We conducted an ScR of the literature and a thematic analysis of the studies included according to the published protocol [ 20 ]. The ScR aimed to answer the first research question, while the thematic analysis aimed to answer the second and third research questions. Spanish experts from various domains of HTA and dHT collaborated throughout the study design and development.

The ScR of the available scientific literature was carried out in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews) guidelines ( Multimedia Appendix 1 ) [ 21 ] and following the recommendations of Peters et al [ 22 ] and Pollock et al [ 23 ].

Ethical Considerations

As this work was an ScR, no ethical board approval was required.

Search Strategy

The search strategy ( Multimedia Appendix 2 ) was designed by an experienced information specialist (author RP-P) in accordance with the research questions and using the validated filter of Ayiku et al [ 24 ] for health apps, adding the terms for concepts related to mHealth, remote care models, AI, digital health, methodological frameworks, and HTA. The strategy was peer-reviewed according to the “Peer Review of Electronic Search Strategies Statement” [ 25 ] by authors JS-F and CM-P and was executed in the following 7 databases, considering the characteristics of each in terms of syntax, controlled vocabulary, and proximity operators: Medline (OVID), CINAHL Plus, Embase, Cochrane Library, Scopus, Web of Science, and TripDatabase. Note that no time, language, or other filters were used.

The identification of relevant studies was complemented with a manual search based on the references in the included studies, as well as the websites of the HTA agencies identified through the web pages of EUnetHTA, the International Network for Agencies for Health Technology Assessment (INAHTA), and Health Technology Assessment International (HTAi). Additionally, a search was conducted in Google Scholar, limiting the results to the first 250 items in order to guarantee the inclusion of all pertinent studies [ 26 ].

Inclusion and Exclusion Criteria

The inclusion criteria used in the reference-screening process were based on the previously detailed research questions and are outlined in Textbox 1 using the Population/Problem, Phenomenon of Interest, Context and Design (PICo-D) format [ 27 , 28 ]. The PICo-D format was used instead of the traditional Population/Problem, Intervention, Comparator, Outcomes, Design (PICO-D) format due to the qualitative nature of the research questions and the characteristics of the phenomenon of interest.

Studies were excluded if they were published before 2011, due to the rapid evolution of dHTs in the past few years, did not describe dimensions or evaluation criteria, or were based on methodological frameworks not intended for the assessment of dHTs (eg, EUnetHTA Core Model 3.0). Likewise, we excluded comments, editorials, letters, conference abstracts, frameworks, or tools focusing on the evaluation of dHTs by users (eg, User version of Mobile App Rating Scale [uMARS]) or documents in languages other than English, Spanish. or Catalan.

Population/problem

Digital health technology assessment (dHTA)

Phenomenon of interest

Specific methodological frameworks for the evaluation of digital health (with special focus on mobile health [mHealth]: non–face-to-face care models and medical devices that integrate artificial intelligence [AI] due the type of technologies mostly assessed in the Spanish National Health System [SNS]) that describe the domains to be evaluated in dHTA

Health technology assessment (HTA)

Methodological guidelines and frameworks, scoping reviews (ScRs), systematic reviews (SRs), consensus documents, and qualitative studies

Reference Screening and Data Extraction

The screening of studies was carried out by authors CM-P and JS-F in 2 phases in accordance with the selection criteria detailed earlier ( Textbox 1 ) and in a single-blind peer review manner. The first phase consisted of screening of the titles and abstracts of the studies identified in the bibliographic search. The second phase consisted of full-text screening of the studies included in the previous phase.

Data extraction was performed by 3 authors (CM-P, RP-P, and JS-F) using the web and desktop versions of ATLAS.ti version 22.0 (Scientific Software Development GmbH) [ 29 ] and the data extraction sheets designed ad hoc for this purpose following the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions [ 30 ].

When disagreements emerged in either of the 2 processes, a consensus was reached between the 3 reviewers (CM-P, RP-P, and JS-F). When a consensus was not possible, a fourth reviewer (author RMV-H) was consulted.

Collecting, Summarizing, and Reporting the Results

A descriptive analysis was carried out to evaluate and report the existing methodological frameworks and their characteristics.

Overview of Methods for Thematic Analysis

The thematic analysis was performed following the recommendations and phases described by Thomas and Harden [ 31 ] to determine HTA dimensions for dHTs: (1) line-by-line text coding, (2) development of descriptive topics, and (3) generation of analytical themes. Both analyses were carried out by 3 authors (CM-P, RP-P, and JS-F) using the web and desktop versions of ATLAS.ti version 22.0 [ 29 ].

Dimensions identified from systematic reviews (SRs) that were derived from primary studies also identified in our systematic search were only counted once in order to avoid duplication of data and risk of bias. It is worth mentioning that the primary studies included in the SRs were not directly analyzed but were analyzed through the findings reported in the SRs.

Study Selection and Characteristics

A total of 3042 studies were retrieved throughout the systematic (n=3023, 99.4%) and the manual (n=19, 0.6%) search. Of these, 2238 (73.6%) studies were identified as unique after removing duplicates.

After title and abstract review, 81 (3.6%) studies were selected for full-text review, of which 26 (32.1%) were finally included in the analysis. The excluded studies and reasons for exclusion are detailed in Multimedia Appendix 3 ; in brief, the reasons for exclusion were phenomenon of interest (n=30, 37%), type of publication (n=15, 18.5%), purpose (n=6, 7.4%), language (n=2, 2.5%), and duplicated information (n=2, 2.5%). The study selection process is outlined in Figure 1 [ 32 ].

Of the 26 (32.1%) studies included in this ScR, 19 (73.1%) were designed as specific methodological frameworks for dHTA [ 16 , 17 , 33 - 47 ], 4 (15.4%) were SRs [ 48 - 51 ], 1 (3.9%) was a report from the European mHealth Hub’s working group on mHealth assessment guidelines [ 52 ], 1 (3.9%) was a qualitative study [ 53 ], and 1 (3.9%) was a viewpoint [ 54 ]. In addition, 3 (11.5%) focused on the assessment of non–face-to-face care models [ 33 - 35 ], 8 (30.8%) on mHealth assessment [ 36 - 40 , 52 , 53 , 55 ], 2 (7.7%) on the assessment of AI technology [ 41 , 54 ], 4 (15.4%) on eHealth [ 42 , 43 , 48 , 50 ], and 9 (34.6%) on the overall assessment of digital health [ 16 , 17 , 44 - 47 , 49 , 51 , 56 ].

Research Question 1: Description of Identified Frameworks for dHTA

The 19 methodological frameworks for dHTA [ 16 , 17 , 33 - 47 ] were from various countries: The majority (n=5, 26.3%) originated in Australia [ 17 , 34 , 38 , 41 , 46 ], followed by 3 (15.8%) from the United States [ 43 , 45 , 56 ] and 2 (10.5%) from Switzerland [ 47 , 55 ]; the remaining 9 (47.4%) frameworks were developed in Afghanistan [ 42 ], Denmark [ 33 ], Scotland [ 35 ], Finland [ 16 ], Ireland [ 36 ], Israel [ 40 ], the United Kingdom [ 37 ], Spain [ 39 ], and Sweden [ 44 ].

The 19 methodological frameworks focused on evaluating various types of technologies. Specifically, 3 (15.8%) of them were designed for assessing non–face-to-face care models [ 33 - 35 ], 6 (31.6%) for mHealth [ 36 - 40 ], and 1 (5.3%) for AI solutions [ 41 ]. The other 9 (47.4%) frameworks addressed eHealth [ 42 , 43 , 56 ] or digital health in general [ 16 , 17 , 44 - 47 ], which encompasses non–face-to-face care models, mHealth, and occasionally AI-based solutions [ 18 ] within its scope. It is pertinent to mention that the differentiation between the methodological frameworks designed for the evaluation of eHealth and those designed for dHTA was based on the specific terminology and descriptions used by the authors of those frameworks.

The structures and characteristics of the analyzed methodological frameworks were considered heterogeneous in terms of evaluation specificity (whether they focused on a global evaluation that encompassed more than 1 domain or dimension or on a specific assessment that addressed only 1 domain or dimension), assessment approach (whether they adopted a phased evaluation, a domain evaluation, or a hybrid of both), and number of domains included. Regarding evaluation specificity, 17 (89.5%) methodological frameworks were classified as global as they covered various aspects or domains within their scope [ 16 , 17 , 33 - 36 , 38 - 47 , 55 , 56 ], while 2 (10.5%) were classified as specific as they concentrated exclusively on 1 element or domain of assessment [ 37 , 46 ]. Regarding the assessment approach, 14 (73.7%) methodological frameworks proposed a domain-based evaluation [ 16 , 17 , 33 , 35 , 36 , 38 - 40 , 43 , 44 , 46 , 55 , 56 ], while 4 (21.1%) proposed a hybrid one (phased and domain based) [ 41 , 42 , 45 , 47 ]; the remaining methodological framework did not fit into any of the previous categories, as it was not structured by domains or phases but by types of risk [ 37 ]. Finally, the number of evaluation domains considered ranged from 1 to 14, with an average of 7. Table 1 outlines the primary features of the included methodological frameworks and provides a thorough breakdown of the domains and dimensions they address.

In contrast, from 3 (75%) [ 49 - 51 ] of the 4 SRs [ 48 - 51 ] and the report from the working group on guidelines for the evaluation of mHealth solutions from the European mHealth Hub [ 52 ], we identified other methodological frameworks and tools focusing on the assessment of dHTs. Specifically, we identified 16 methodological frameworks or tools focusing on the evaluation of non–face-to-face care models [ 57 - 72 ], along with 37 for the evaluation of mHealth [ 10 , 52 , 73 - 95 ], 11 for the evaluation of eHealth [ 96 - 107 ], and 17 for the evaluation of dHTs in general [ 108 - 124 ]. Additionally, 5 (26.3%) [ 33 , 34 , 36 , 37 , 42 ] of the 19 methodological frameworks included in this ScR were also identified and analyzed in 1 or more of the 4 literature synthesis documents [ 49 - 52 ]. It is important to note that the difference between the frameworks we retrieved through our systematic search and those identified in the 4 SRs is the result of the narrower perspective we adopted, focusing exclusively on frameworks directly relevant to the HTA field, in line with the aims of our study. In Multimedia Appendix 4 , we provide a more detailed explanation of the methodological frameworks included in the studies mentioned earlier [ 19 , 49 - 52 , 57 - 73 , 75 - 135 ].

a ScR: scoping review.

b mHealth: mobile health.

c N/A: not applicable.

d AI: artificial intelligence.

e dHT: digital health technology.

Research Question 2: Domains and Dimensions Being Considered in dHTA

The 26 (32.1%) studies included encompassed a broad range of items to consider in dHTA and often used diverse expressions for analogous concepts. We reduced this heterogeneity through our thematic analysis according to the recommendations and phases described by Thomas and Harden [ 31 ].

In this sense, in the first phase of thematic analysis, we identified and coded 176 units of meaning (coded as provisional codes) that represented different items (domains or dimensions) of the assessment. These units were then grouped into 86 descriptive themes (second phase), which were further refined into 61 analytical themes that captured the key concepts and relationships between them (third phase). Lastly, the 61 analytical themes were arranged in a 3-level vertical hierarchy based on the evidence: level 1 (12 domains), level 2 (38 dimensions), and level 3 (11 subdimensions). We used the term “domain” to refer to a distinct area or topic of evaluation that is integral to the assessment of the technology in question. A domain may encompass multiple related concepts or dimensions that are relevant to the evaluation. Each dimension, in turn, represents a specific aspect of evaluation that belongs to the domain and contributes to an understanding of its overall significance. Finally, a subdimension refers to a partial element of a dimension that facilitates its analysis. By using these terms, we aimed to provide a clear, rigorous, and comprehensive framework for conducting HTA.

Table 2 displays the 61 analytical themes in descending order of coding frequency, aligned with the hierarchy derived from the data analysis. Additionally, the table specifies the intervention modalities or dHTs that correspond to each code and lists the studies from which each code originated. The network of relationships among the codes can be found in Multimedia Appendix 5 .

a dHT: digital health technology.

c AI: artificial intelligence.

d N/A: not applicable.

Research Question 3: Variability of Domains and Dimensions Among Technologies

Our thematic analysis revealed a significant degree of variability and heterogeneity in the number and type of domains and dimensions considered by the methodological frameworks.

In terms of numbers, the variability was quite pronounced when we compared frameworks addressing different types of dHTs. For instance, the thematic analysis of frameworks for assessing telemedicine only identified 9 (75%) domains and 6 (15.8%) dimensions; instead, in frameworks for assessing mHealth, we identified 10 (83.3%) domains, 20 (52.6%) dimensions, and 6 (54.5%) subdimensions, and in frameworks for assessing AI, we identified 8 (66.7%) different domains, 7 (18.4%) different dimensions, and 6 (54.5%) subdimensions.

In terms of the types of domains considered, certain dimensions and domains were identified as more distinctive for one kind of dHT than for another. For instance, clinical efficacy and effectiveness, technical safety, economic evaluation, and user experience were relevant for the evaluation of models of nonpresential health care and mHealth but not for AI. In contrast, there were specific dimensions and domains of mHealth that were not considered in the evaluation of non–face-to-face health care or AI, such as postmarketing monitoring, scientific basis, technical evaluation and validation, user control and self-determination, accessibility, content and adequacy of information, and data interoperability and integration. Finally, specific methodological frameworks for the evaluation of AI included dimensions such as technical aspects, adoption, use, integration, generalizability, reproducibility, and interpretability, which were not considered in the evaluation of telemedicine or mHealth. In conclusion, greater clarity and structuring in the presentation of these ideas are required to facilitate their understanding and assimilation.

Proposal for Domains, Dimensions, and Subdimensions for dHTA

These findings led to the development of a proposed methodological framework for dHTA, which comprises domains, dimensions, and subdimensions. These evaluation items were established objectively based on thematically analyzed evidence, without incorporating the researcher’s perspective. Consequently, the proposal for domains, dimensions, and subdimensions emerged from the literature and represents the entirety of identified evaluation domains, dimensions, and subdimensions (n=61). Figure 2 presents a visual representation of the proposed framework comprising 12 domains, 38 dimensions, and their corresponding 11 subdimensions. Notably, the figure highlights certain domains, dimensions, and subdimensions that are particularly relevant to the evaluation of non–face-to-face care models, mHealth, and AI according to the evidence.

Principal Findings

In recent years, the interest in digital health has increased significantly, giving rise to a myriad of available technologies. This has brought about a profound transformation in health care systems, fundamentally changing the provision and consumption of health care services [ 9 ]. However, despite these advancements, the shift toward digital health has been accompanied by challenges. One such challenge is the emergence of a plethora of short-lived implementations and an overwhelming diversity of digital tools, which has created a need for careful evaluation and analysis of the benefits and drawbacks of these technologies [ 4 ].

In this context, our ScR aimed to identify the methodological frameworks used worldwide for the assessment of dHTs; determine what domains are considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

Throughout the ScR, we identified a total of 95 methodological frameworks and tools, of which 19 [ 16 , 17 , 33 - 47 ] were directly identified through a systematic search and 75 were indirectly identified through 4 SRs [ 49 - 52 ]. The difference in the number of methodological frameworks identified through the ScR and the 4 evidence synthesis documents [ 49 - 52 ] is attributed to the inclusion of keywords related to the concept of HTA in the search syntax, the exclusion of methodological frameworks published prior to 2011 during the screening process, and the differences in perspectives used for the development of this paper compared to the 4 evidence synthesis documents mentioned earlier. In this sense, these 4 documents [ 49 - 52 ] have analyzed methodological frameworks and tools aimed at evaluating digital health that have not been developed from an HTA perspective despite the authors analyzing them as such. For example, von Huben et al. [ 51 ] included in their analysis the Consolidated Standards of Reporting Trials (CONSORT)-EHEALTH tool [ 97 ], which aims to describe the information that should be reported in papers and reports that focus on evaluating web- and mHealth-based interventions; Koladas et al [ 49 ] included the mobile health evidence reporting and assessment (mERA) checklist [ 73 ], which aims to determine the information that should be reported in trials evaluating mHealth solutions; and the European mHealth Hub document [ 52 ] includes the Isys Score, which is for cataloguing apps for smartphones.

However, as detailed in the Results section, some of the methodological frameworks identified through the ScR were characterized by the authors themselves as being specific for evaluating certain types of dHTs (eg, non–face-to-face care models, mHealth), presenting certain differences according to each typology. It is important to note that the differentiation among various types of dHTs, as described throughout this paper and commonly used in the field of digital health, cannot always be made in a precise and exclusive manner [ 136 ]. This is because a technology often can be classified in more than 1 category. For instance, an mHealth solution may use AI algorithms, while simultaneously being integrated into a non–face-to-face care model [ 137 ]. In this context, future research should consider using alternative taxonomies or classification methods that are based on the intended purpose of the technology, such as those proposed by NICE in the updated version of the Evidence Standards Framework [ 18 ] or the new digital health interventions system classification put forward by WHO [ 138 ].

After conducting a thematic analysis of the 26 included studies, we observed that various methodological frameworks include a set of evaluation items, referred to as domains, dimensions, or criteria. These items primarily focus on the safety; effectiveness; technical aspects; economic impact; and ethical, legal, and social consequences of dHTs. However, there is significant heterogeneity among these frameworks in terms of the way they refer to the evaluation items, the quantity and depth of their description, the degree of granularity, and the proposed evaluation methods, especially when comparing frameworks that focus on different types of dHTs. Despite this heterogeneity, most methodological frameworks consider evaluation items related to the 9 domains described by the HTA Core Model of EUnetHTA, while some frameworks propose additional evaluation elements, such as usability [ 16 , 44 , 45 , 47 , 49 , 56 ], privacy [ 39 - 41 , 44 , 52 , 55 ], and technical stability [ 16 , 38 , 47 , 49 , 52 ] among others. These findings are consistent with earlier research [ 50 , 51 ].

In addition, through the thematic analysis, the heterogeneity identified among the different methodological frameworks included in this ScR was reduced to a total of 61 analytical themes related to various evaluation elements that were arranged in a 3-level vertical hierarchy based on the evidence: level 1 (12 domains), level 2 (38 dimensions), and level 3 (11 subdimensions). At this point, it is pertinent to note that although from the researchers’ perspective, some dimensions could have been classified under different domains (eg, responsibility under ethical aspects) or seen as essential for other kinds of dHTs, an effort was made to maintain the highest degree of objectivity possible. It is for this reason that privacy issues were not described as essential for non–face-to-face care models and why the dimension of accessibility was categorized within the domains of human and sociocultural aspects and technical aspects. This categorization was made because some of the methodological frameworks analyzed associated it with sociocultural elements (eg, evaluating whether users with functional diversity can access the technology and have sufficient ability to use it as expected), while others linked it to technical elements (eg, adequacy of the elements, options, or accessibility functionalities that the system incorporates according to the target audience) [ 16 , 52 ].

The ScR and thematic analysis conducted in this study led to a proposal for a methodological framework for dHTA. This framework was further developed using additional methodologies, such as consensus workshops by the Agency for Health Quality and Assessment of Catalonia (AQuAS), in collaboration with all agencies of RedETS, commissioned by the Ministry of Health of Spain. The final framework is a specific methodological tool for the assessment of dHTs, aimed at describing the domains and dimensions to be considered in dHTA and defining the evidence standards that such technologies must meet based on their associated risk level. The proposed methodological framework enables the assessment of a wide range of dHTs, mainly those classified as medical devices according to the Regulation (EU) 2017/745 for medical devices [ 139 ] and Regulation (EU) 2017/746 for in vitro diagnostic medical devices, although it can be adapted to assess dHTs not classified as medical devices [ 140 ]. Unlike existing frameworks, it establishes a clear link between the identified domains and dimensions and the evidence standards required for dHTs to meet. This approach will enhance the transparency and consistency of dHTAs and support evidence-based decision-making. The final document was published from November 2023 onward and is available on the RedETS website as well as on the main web page of AQuAS in the Spanish language [ 141 ]. From the first week of February, the respective websites have hosted an English version of this document [ 141 ], which also is accessible in the INAHTA database. In addition, the Spanish and English versions of the document will be periodically reviewed and, if necessary, adapted to align with emerging technologies and changes in legislation.

Limitations

Although this ScR was conducted in accordance with the PRISMA-ScR guidelines ( Multimedia Appendix 1 ) and following the recommendations of Peters et al [ 22 ] and Pollock et al [ 23 ], there were some limitations. First, the search incorporated a block of keywords related to the concept of HTA (see Multimedia Appendix 1 ) due to the perspective of our ScR, which may have limited the retrieval of some studies to meet the study objective. However, this limitation was compensated for by the analysis of the 3 SRs and the report of the working group on guidelines for the evaluation of mHealth solutions of the European mHealth Hub. Second, much of the literature related to HTA is gray literature and only published on the websites of the authoring agencies. Despite efforts to address this limitation through expert input and a comprehensive search of the websites of the world’s leading agencies, it is possible that certain studies were not identified. Third, the quality and limitations of the analysis conducted by the authors of methodological frameworks and tools included in SRs may have had an impact on the indirect thematic analysis. Therefore, it is possible that some data could have been omitted or not considered during this process. Fourth, the focus on dHTs encompassed within the 3 previously mentioned categories (mHealth, non–face-to-face care models, and medical devices that integrate AI) may have influenced the outcomes of the thematic analysis conducted. Fifth, only methodological frameworks written in Catalan, Spanish, and English were included.

Comparison With Prior Work

To the best of our knowledge, this is the first ScR to examine the methodological frameworks for dHTA, followed by a thematic analysis with the aim of proposing a new comprehensive framework that incorporates the existing literature in an objective manner and enables the assessment of various technologies included under the concept of digital health. In this sense, existing SRs and other evidence synthesis documents have only analyzed the literature and reported the results in a descriptive manner [ 36 , 48 , 49 , 51 , 56 , 125 , 126 ]. Furthermore, this ScR also considered, in addition to scientific literature, gray literature identified by searching the websites of the agencies, thus covering some limitations of previous reviews [ 50 ]. Moreover, this review was carried out from the perspective of HTA, addressing a clear need expressed by HTA agencies [ 16 ].

Future research should aim to identify what domains and dimensions are relevant at the different stages of the technology life cycle, to establish or develop a standardized set of outcomes for assessing or reporting each domain, and to evaluate the effectiveness and usefulness of the existing methodological frameworks for the different intended users [ 50 , 142 ]. Moreover, future research should aim to determine the specific evaluation criteria that ought to be considered based on the level of risk associated with different types of technologies [ 51 ].

Our ScR revealed a total of 102 methodological frameworks and tools designed for evaluating dHTs, with 19 being directly identified through a systematic search and 83 through 4 evidence synthesis documents. Only 19 of all the identified frameworks were developed from the perspective of HTA. These frameworks vary in assessment items, structure, and specificity, and their proven usefulness in practice is scarce.

The thematic analysis of the 26 studies that met the inclusion criteria led to the identification and definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered when evaluating dHTs. Building on our results, a methodological framework for dHTA was proposed.

Acknowledgments

We acknowledge Benigno Rosón Calvo (Servicio Gallego de Salud [SERGAS]), Carme Carrion (Universitat Oberta de Catalunya [UOC]), Carlos A Molina Carrón (Dirección General de Salud Digital y Sistemas de Información para el SNS. Ministerio de Sanidad, Gobierno de España), Carme Pratdepadua (Fundació Tic Salut i Social [FTSS]), Celia Muñoz (Instituto Aragonés de Ciencias de la Salud [IACS]), David Pijoan (Biocat, BioRegió de Catalunya), Felip Miralles (Eurecat – Centre Tecnològic de Catalunya), Iñaki Guiterrez Ibarluzea (Osasun Teknologien Ebaluazioko Zerbitzua [Osteba]), Janet Puñal Riobóo (Unidad de Asesoramiento Científico-técnico [avalia-t], Agencia Gallega para la Gestión del Conocimiento en Salud [ACIS]), Jordi Piera-Jiménez (Àrea de Sistemes d’Informació del Servei Català de la Salut [CatSalut]), Juan Antonio Blasco (Evaluación de Tecnologías Sanitarias de Andalucía [AETSA]), Liliana Arroyo Moliner (Direcció General de Societat Digital, Departament d’Empresa i Treball de la Generalitat de Catalunya), Lilisbeth Perestelo-Perez (Servicio de Evaluación del Servicio Canario de la Salud [SESCS]), Lucía Prieto Remón (IACS), Marifé Lapeña (Dirección General de Salud Digital y Sistemas de Información para el SNS. Ministerio de Sanidad, Gobierno de España), Mario Cárdaba (Insituto de Salud Carlos III [ISCIII]), Montserrat Daban (Biocat, BioRegió de Catalunya), Montserrat Moharra Frances (Agència de Qualitat i Avaluació Sanitàries de Catalunya), and Oscar Solans (CatSalut) for reviewing the protocol of this scoping review (ScR) and the ScR.

This research was framed within the budget of the work plan of the Spanish Network of Health Technology Assessment Agencies, commissioned by the General Directorate of Common Portfolio of Services of the National Health System and Pharmacy.

Authors' Contributions

JS-F and CM-P were responsible for conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, and visualization. RP-P handled conceptualization, methodology, formal analysis, investigation, resources, and writing—original draft. RMV-H handled conceptualization, writing—review and editing, supervision, and project administration.

Conflicts of Interest

None declared.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR) checklist [ 21 ].

Search strategies for each database.

References excluded at the full-text screening stage.

Methodological frameworks included in systematic reviews.

Network of relationships among the codes.

High-resolution image of Figure 2.

• Avanzas P, Pascual I, Moris C. The great challenge of the public health system in Spain. J Thorac Dis. May 2017;9(Suppl 6):S430-S433. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grubanov BS, Ghio D, Goujon A, Kalantaryan S, Belmonte M, Scipioni M. Health and long-term care workforce: demographic challenges and the potential contribution of migration and digital technology. Luxembourg. European Commission; 2021;2045-2322.
• World Health Organization (WHO). Health 2020: a European policy framework and strategy for the 21st century. Denmark. WHO; 2013.
• World Health Organization (WHO). WHO guideline: recommendations on digital interventions for health system strengthening. Geneva. WHO; Jun 6, 2019.
• Scholz N. Addressing health inequalities in the European Union. Luxembourg. European Parliamentary Research Service; 2020.
• Kickbusch I, Agrawal A, Jack A, Lee N, Horton R. Governing health futures 2030: growing up in a digital world—a joint The Lancet and Financial Times Commission. Lancet. Oct 2019;394(10206):1309. [ CrossRef ]
• Reeves JJ, Ayers JW, Longhurst CA. Telehealth in the COVID-19 era: a balancing act to avoid harm. J Med Internet Res. Feb 01, 2021;23(2):e24785. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Richardson S, Lawrence K, Schoenthaler AM, Mann D. A framework for digital health equity. NPJ Digit Med. Aug 18, 2022;5(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• European Union. State of health in the EU: companion report 2021. Luxembourg. European Union; 2022.
• LATITUD: Anàlisi comparativa de models d'atenció no presencial en l'àmbit de la salut. TIC Salut Social. 2020. URL: https://ticsalutsocial.cat/wp-content/uploads/2021/07/latitud.pdf [accessed 2024-03-28]
• Global strategy on digital health 2020-2025. World Health Organization. Aug 18, 2021. URL: https://www.who.int/publications/i/item/9789240020924 [accessed 2024-03-28]
• Ming J, He Y, Yang Y, Hu M, Zhao X, Liu J, et al. Health technology assessment of medical devices: current landscape, challenges, and a way forward. Cost Eff Resour Alloc. Oct 05, 2022;20(1):54. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Rourke B, Oortwijn W, Schuller T. The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care. May 13, 2020;36(3):187-190. [ CrossRef ]
• EUnetHTA Joint Action 2 Work Package 8. HTA Core Model ® version 3.0 2016. EUnetHTA. 2016. URL: https://www.eunethta.eu/wp-content/uploads/2018/03/HTACoreModel3.0-1.pdf [accessed 2024-03-28]
• Puñal-Riobóo J, Baños, Varela LL, Castillo MM, Atienza MG, Ubago PR. Guía para la elaboración y adaptación de informes rápidos. Santiago de Compostela, Madrid. Agencia Gallega para la Gestión del Conocimientoto en Salud. Unidad de Asesoramiento Científico-técnico, avalia-t. Ministerio de Sanidad, Servicios Sociales e Igualdad; 2016.
• Haverinen J, Keränen N, Falkenbach P, Maijala A, Kolehmainen T, Reponen J. Digi-HTA: health technology assessment framework for digital healthcare services. FinJeHeW. Nov 02, 2019;11(4):326-341. [ CrossRef ]
• Hussain MS, Silvera-Tawil D, Farr-Wharton G. Technology assessment framework for precision health applications. Int J Technol Assess Health Care. May 26, 2021;37(1):e67. [ CrossRef ]
• National Institute for Health and Care Excellence (NICE). Evidence standards framework (ESF) for digital health technologies. Contract no.: 30 de septiembre. London. NICE; 2022.
• Evidence standards framework (ESF) for digital health technologies. National Institute for Health and Care Excellence. URL: https:/​/www.​nice.org.uk/​about/​what-we-do/​our-programmes/​evidence-standards-framework-for-digital-health-technologies [accessed 2024-03-28]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo RM. Methodological frameworks and dimensions to be taken into consideration in digital health technology assessment: protocol for a scoping review. JMIR Res Protoc. Oct 11, 2022;11(10):e39905. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peters M, Marnie C, Tricco A, Pollock D, Munn Z, Alexander L. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119-2126. [ CrossRef ]
• Pollock D, Davies EL, Peters MDJ, Tricco AC, Alexander L, McInerney P, et al. Undertaking a scoping review: a practical guide for nursing and midwifery students, clinicians, researchers, and academics. J Adv Nurs. Apr 04, 2021;77(4):2102-2113. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ayiku L, Hudson T, Glover S, Walsh N, Adams R, Deane J, et al. The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps. Int J Technol Assess Health Care. Oct 27, 2020;37(1):e16. [ CrossRef ]
• McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. Jul 2016;75:40-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. Dec 06, 2017;6(1):245. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc. 2015;13(3):179-187. [ CrossRef ]
• Systematic reviews - research guide. Defining your review question. Murdoch University. 2023. URL: https://libguides.murdoch.edu.au/systematic [accessed 2024-03-28]
• Scientific Software Development GmbH. Atlas.ti qualitative data analysis. 22.0 ed. Berlin. Scientific Software Development GmbH; 2021.
• Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane handbook for systematic reviews of interventions version 6.3. London. Cochrane; 2022.
• Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. Jul 10, 2008;8(1):45. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. Mar 2021;18(3):e1003583. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kidholm K, Ekeland AG, Jensen LK, Rasmussen J, Pedersen CD, Bowes A, et al. A model for assessment of telemedicine applications: MAST. Int J Technol Assess Health Care. Jan 23, 2012;28(1):44-51. [ CrossRef ]
• Nepal S, Li J, Jang-Jaccard J, Alem L. A framework for telehealth program evaluation. Telemed J E Health. Apr 2014;20(4):393-404. [ CrossRef ] [ Medline ]
• Measuring the impact of telehealth and telecare: SCTT toolkit. Glasgow. SCTT; 2013.
• Caulfield B, Reginatto B, Slevin P. Not all sensors are created equal: a framework for evaluating human performance measurement technologies. NPJ Digit Med. Feb 14, 2019;2(1):7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lewis TL, Wyatt JC. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. Sep 15, 2014;16(9):e210. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Moshi MR, Tooher R, Merlin T. Development of a health technology assessment module for evaluating mobile medical applications. Int J Technol Assess Health Care. May 18, 2020;36(3):252-261. [ CrossRef ]
• Puigdomènech E, Poses-Ferrer E, Espallargues CM, Blasco AJ, Varela LL, Paz VL. Evaluación de tecnología basada en mSalud para aplicaciones móviles. Madrid. Ministerio de Sanidad; 2020.
• Henson P, David G, Albright K, Torous J. Deriving a practical framework for the evaluation of health apps. Lancet Digit Health. Jun 2019;1(2):e52-e54. [ CrossRef ]
• Reddy S, Rogers W, Makinen V, Coiera E, Brown P, Wenzel M, et al. Evaluation framework to guide implementation of AI systems into healthcare settings. BMJ Health Care Inform. Oct 12, 2021;28(1):e100444. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Khoja S, Durrani H, Scott RE, Sajwani A, Piryani U. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed J E Health. Jan 2013;19(1):48-53. [ CrossRef ] [ Medline ]
• Sockolow P, Bowles K, Rogers M. Health information technology evaluation framework (HITREF) comprehensiveness as assessed in electronic point-of-care documentation systems evaluations. Amsterdam. IOS Press; 2015.
• Kowatsch T, Otto L, Harperink S, Cotti A, Schlieter H. A design and evaluation framework for digital health interventions. it - Inf Technol. Nov 2019;61(5):253-263. [ CrossRef ]
• Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB. Digital health: a path to validation. NPJ Digit Med. May 13, 2019;2(1):38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Milosevic Z. Ethics in digital health: a deontic accountability framework. 2019. Presented at: IEEE 23rd International Enterprise Distributed Object Computing Conference (EDOC); October 28-31, 2019; Paris, France. [ CrossRef ]
• World Health Organization (WHO). Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Geneva. WHO; 2016.
• Enam A, Torres-Bonilla J, Eriksson H. Evidence-based evaluation of eHealth interventions: systematic literature review. J Med Internet Res. Nov 23, 2018;20(11):e10971. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kolasa K, Kozinski G. How to value digital health interventions? A systematic literature review. Int J Environ Res Public Health. Mar 23, 2020;17(6):2119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vis C, Bührmann L, Riper H, Ossebaard HC. Health technology assessment frameworks for eHealth: a systematic review. Int J Technol Assess Health Care. Apr 16, 2020;36(3):204-216. [ CrossRef ]
• von Huben A, Howell M, Howard K, Carrello J, Norris S. Health technology assessment for digital technologies that manage chronic disease: a systematic review. Int J Technol Assess Health Care. May 26, 2021;37(1):e66. [ CrossRef ]
• Report of the Working Group on mhealth assessment guidelines. European Commission. 2021. URL: https://digital-strategy.ec.europa.eu/en/library/report-working-group-mhealth-assessment-guidelines [accessed 2024-03-28]
• Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, et al. Mobile health technology evaluation: the mhealth evidence workshop. Am J Prev Med. Aug 2013;45(2):228-236. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alami H, Lehoux P, Auclair Y, de Guise M, Gagnon M, Shaw J, et al. Artificial intelligence and health technology assessment: anticipating a new level of complexity. J Med Internet Res. Jul 07, 2020;22(7):e17707. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vokinger KN, Nittas V, Witt CM, Fabrikant SI, von Wyl V. Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Med Wkly. May 04, 2020;150(1920):w20282. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Baumel A, Muench F. Heuristic evaluation of ehealth interventions: establishing standards that relate to the therapeutic process perspective. JMIR Ment Health. Jan 13, 2016;3(1):e5. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alfonzo A, Huerta M, Wong S, Passariello G, Diaz M, La CA, et al. Design of a methodology for assessing an electrocardiographic telemonitoring system. Annu Int Conf IEEE Eng Med Biol Soc. 2007;2007:3729-3732. [ CrossRef ]
• Bashshur R, Shannon G, Sapci H. Telemedicine evaluation. Telemed J E Health. Jun 2005;11(3):296-316. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Beintner I, Vollert B, Zarski A, Bolinski F, Musiat P, Görlich D, et al. Adherence reporting in randomized controlled trials examining manualized multisession online interventions: systematic review of practices and proposal for reporting standards. J Med Internet Res. Aug 15, 2019;21(8):e14181. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brear M. Evaluating telemedicine: lessons and challenges. Health Inf Manag. Jul 21, 2006;35(2):23-31. [ CrossRef ] [ Medline ]
• DeChant HK, Tohme WG, Mun SK, Hayes WS, Schulman KA. Health systems evaluation of telemedicine: a staged approach. Telemed J. Jan 1996;2(4):303-312. [ CrossRef ] [ Medline ]
• Giansanti D, Morelli S, Macellari V. Telemedicine technology assessment part II: tools for a quality control system. Telemed J E Health. Apr 2007;13(2):130-140. [ CrossRef ] [ Medline ]
• Giansanti D, Morelli S, Macellari V. Telemedicine technology assessment part I: setup and validation of a quality control system. Telemed J E Health. Apr 2007;13(2):118-129. [ CrossRef ] [ Medline ]
• Grigsby J, Brega AG, Devore PA. The evaluation of telemedicine and health services research. Telemed J E Health. Jun 2005;11(3):317-328. [ CrossRef ] [ Medline ]
• Hailey D, Jacobs P, Simpson J, Doze S. An assessment framework for telemedicine applications. J Telemed Telecare. Jun 23, 1999;5(3):162-170. [ CrossRef ] [ Medline ]
• Ohinmaa A, Hailey D, Roine R. Elements for assessment of telemedicine applications. Int J Technol Assess Health Care. Jun 30, 2001;17(2):190-202. [ CrossRef ] [ Medline ]
• Rajan B, Tezcan T, Seidmann A. Service systems with heterogeneous customers: investigating the effect of telemedicine on chronic care. Manag Sci. Mar 2019;65(3):1236-1267. [ CrossRef ]
• Sisk JE, Sanders JH. A proposed framework for economic evaluation of telemedicine. Telemed J. Jan 1998;4(1):31-37. [ CrossRef ] [ Medline ]
• Zissman K, Lejbkowicz I, Miller A. Telemedicine for multiple sclerosis patients: assessment using Health Value Compass. Mult Scler. Apr 30, 2012;18(4):472-480. [ CrossRef ] [ Medline ]
• Grustam AS, Vrijhoef HJM, Koymans R, Hukal P, Severens JL. Assessment of a business-to-consumer (B2C) model for telemonitoring patients with chronic heart failure (CHF). BMC Med Inform Decis Mak. Oct 11, 2017;17(1):145. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hebert M. Telehealth success: evaluation framework development. Stud Health Technol Inform. 2001;84(Pt 2):1145-1149. [ Medline ]
• Rojahn K, Laplante S, Sloand J, Main C, Ibrahim A, Wild J, et al. Remote monitoring of chronic diseases: a landscape assessment of policies in four European countries. PLoS One. May 19, 2016;11(5):e0155738. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, et al. WHO mHealth Technical Evidence Review Group. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ. Mar 17, 2016;352:i1174. [ CrossRef ] [ Medline ]
• Safety and quality strategy in mobile health apps. Complete list of recommendations on design, use and assessment of health apps. Agencia de Calidad Sanitaria de Andalucía. 2012. URL: http://www.calidadappsalud.com/en/listado-completo-recomendaciones-app-salud/ [accessed 2023-07-17]
• The app evaluation model. American Psychiatric Association Initiative. 2022. URL: https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/the-app-evaluation-model [accessed 2024-03-28]
• Gdd AppStore. Association of Regional Public Health Services (GGD) and Regional Medical Emergency Preparedness and Planning (GHOR). 2016. URL: https://tinyurl.com/58th4p4w [accessed 2023-07-18]
• AppQ: quality criteria core set for more quality transparency in digital health applications. Bertelsmann Stiftung. Oct 29, 2019. URL: https://www.bertelsmann-stiftung.de/de/publikationen/publikation/did/appq [accessed 2024-03-28]
• Bradway M, Carrion C, Vallespin B, Saadatfard O, Puigdomènech E, Espallargues M, et al. mHealth assessment: conceptualization of a global framework. JMIR Mhealth Uhealth. May 02, 2017;5(5):e60. [ FREE Full text ] [ CrossRef ] [ Medline ]
• PAS 277:2015 - health and wellness apps. Quality criteria across the life cycle. British Standards Institution. URL: https:/​/knowledge.​bsigroup.com/​products/​health-and-wellness-apps-quality-criteria-across-the-life-cycle-code-of-practice/​standard [accessed 2023-07-18]
• MindApps. Centre for Telepsychiatry in the Region of Southern Denmark. 2017. URL: https://mindapps.dk/ [accessed 2023-07-18]
• Dick S, O'Connor Y, Thompson MJ, O'Donoghue J, Hardy V, Wu TJ, et al. Considerations for improved mobile health evaluation: retrospective qualitative investigation. JMIR Mhealth Uhealth. Jan 22, 2020;8(1):e12424. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Federal Institute for Drugs and Medical Devices (BfArM). The fast-track process for digital health applications (diga) according to section 139e sgb v. A guide for manufacturers, service providers and users. Bonn, Germany. Federal Institute for Drugs and Medical Devices; 2020.
• Gorski I, Bram JT, Sutermaster S, Eckman M, Mehta K. Value propositions of mHealth projects. J Med Eng Technol. Aug 12, 2016;40(7-8):400-421. [ CrossRef ] [ Medline ]
• Hogaboam L, Daim T. Technology adoption potential of medical devices: the case of wearable sensor products for pervasive care in neurosurgery and orthopedics. Health Policy Technol. Dec 2018;7(4):409-419. [ CrossRef ]
• Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. Apr 05, 2019;2(4):e192542. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maar MA, Yeates K, Perkins N, Boesch L, Hua-Stewart D, Liu P, et al. A framework for the study of complex mhealth interventions in diverse cultural settings. JMIR Mhealth Uhealth. Apr 20, 2017;5(4):e47. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McMillan B, Hickey E, Patel MG, Mitchell C. Quality assessment of a sample of mobile app-based health behavior change interventions using a tool based on the National Institute of Health and Care Excellence behavior change guidance. Patient Educ Couns. Mar 2016;99(3):429-435. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Health and welness apps: new international guidelines to help to sort the best form the rest. NEN. URL: https://www.nen.nl/en/health-and-welness-apps [accessed 2024-03-28]
• Continua design guidelines. Personal Connected Health Alliance. 2019. URL: https://www.pchalliance.org/continua-design-guidelines [accessed 2023-07-18]
• Philpott D, Guergachi A, Keshavjee K. Design and validation of a platform to evaluate mhealth apps. Stud Health Technol Inform. 2017;235:3-7. [ Medline ]
• Ruck A, Wagner BS, Lowe C. Second draft of guidelines. EU guidelines on assessment of the reliability of mobile health applications. European Commission, Directorate-General of Communications Networks, Content & Technology. Luxembourg; 2016.
• Sax M, Helberger N, Bol N. Health as a means towards profitable ends: mhealth apps, user autonomy, and unfair commercial practices. J Consum Policy. May 22, 2018;41(2):103-134. [ CrossRef ]
• Wyatt JC. How can clinicians, specialty societies and others evaluate and improve the quality of apps for patient use? BMC Med. Dec 03, 2018;16(1):225. [ FREE Full text ] [ CrossRef ] [ Medline ]
• IRBs could address ethical issues related to tracking devices: mobile devices raise new concerns. IRB Advisor. Nov 1, 2017. URL: https:/​/www.​reliasmedia.com/​articles/​141589-irbs-could-address-ethical-issues-related-to-tracking-devices [accessed 2024-03-28]
• App check. Zentrum für Telematik und Telemedizin. URL: https://ztg-nrw.de/ [accessed 2024-03-28]
• Bergmo TS. How to measure costs and benefits of ehealth interventions: an overview of methods and frameworks. J Med Internet Res. Nov 09, 2015;17(11):e254. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. Dec 31, 2011;13(4):e126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shaw NT. 'CHEATS': a generic information communication technology (ICT) evaluation framework. Comput Biol Med. May 2002;32(3):209-220. [ CrossRef ] [ Medline ]
• Brown M, Shaw N. Evaluation practices of a major Canadian telehealth provider: lessons and future directions for the field. Telemed J E Health. Oct 2008;14(8):769-774. [ CrossRef ] [ Medline ]
• Casper GR, Kenron DA. A framework for technology assessment: approaches for the selection of a home technology device. Clin Nurse Spec. 2005;19(4):170-174. [ CrossRef ] [ Medline ]
• Sitting D, Kahol K, Singh H. Sociotechnical evaluation of the safety and effectiveness of point-of-care mobile computing devices: a case study conducted in India. Stud Health Technol Inform. 2013;192:515-519. [ CrossRef ]
• Haute Autorité de Santé. Good practice guidelines on health apps and smart devices (mobile health or mhealth). Paris. Haute Autorité de Santé; Nov 7, 2016.
• Health Information and Quality Authority. International review of consent models for the collection, use and sharing of health information. Dublin. Health Information and Quality Authority; 2020.
• Jurkeviciute M. Planning of a holistic summative ehealth evaluation: the interplay between standards and reality PhD Thesis. Gotherburg. Chalmers University of Technology; 2018.
• Vimarlund V, Davoody N, Koch S. Steps to consider for effective decision making when selecting and prioritizing eHealth services. Stud Health Technol Inform. 2013;192:239-243. [ Medline ]
• Currie WL. TEMPEST: an integrative model for health technology assessment. Health Policy Technol. Mar 2012;1(1):35-49. [ CrossRef ]
• Eivazzadeh S, Anderberg P, Larsson TC, Fricker SA, Berglund J. Evaluating health information systems using ontologies. JMIR Med Inform. Jun 16, 2016;4(2):e20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Our data-driven future in healthcare: people and partnerships at the heart of health related technologies. Academy of Medical Sciences. Nov 2018. URL: https://acmedsci.ac.uk/file-download/74634438 [accessed 2024-03-28]
• Australian Commission on Safety and Quality in Healthcare. National safety and quality digital mental health standards - consultation draft. Australia. Australian Commission on Safety and Quality in Healthcare; 2020.
• A guide to good practice for digital and data-driven health technologies. London. Department of Health & Social Care; 2021.
• Drury P, Roth S, Jones T, Stahl M, Medeiros D. Guidance for investing in digital health. In: Sustainable Development Working Papers. Mandaluyong, the Philippines. Asian Development Bank; May 2018.
• European Commission. Synospis report. Consultation: transformation health and care in the digital single market. Luxembourg. European Commission; 2018.
• Federal Ministry of Health. Regulation on the procedure and requirements for testing the eligibility for reimbursement of digital health applications in the statutory public health insurance (Digital Health Applications Ordinance - DiGAV). Alemania. Federal Ministry of Health; 2020.
• Haute Autorité de Santé. Guide to the specific features of clinical evaluation of a connected medical device (CMD) in view of its application for reimbursement. Paris. Haute Autorité de Santé; 2019.
• How we assess health apps and digital tools. NHS Digital. 2019. URL: https:/​/digital.​nhs.uk/​services/​nhs-apps-library/​guidance-forhealth-app-developers-commissioners-and-assessors/​how-we-assess-healthapps-and-digital-tools [accessed 2023-07-17]
• Lennon MR, Bouamrane M, Devlin AM, O'Connor S, O'Donnell C, Chetty U, et al. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. Feb 16, 2017;19(2):e42. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McNamee P, Murray E, Kelly MP, Bojke L, Chilcott J, Fischer A, et al. Designing and undertaking a health economics study of digital health interventions. Am J Prev Med. Nov 2016;51(5):852-860. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Haute Autorité de Santé. Medical Device and Health Technology Evaluation Committee (CNEDiMTS*). Paris. Haute Autorité de Santé; 2019.
• Draft guidelines for preparing assessment reports for the Medical Services Advisory Committee: draft version 4.0. Australian Government Department of Health and Aged Care. Aug 2020. URL: https:/​/consultations.​health.gov.au/​technology-assessment-access-division/​msac-guidelines-review-consultation/​user_uploads/​draft-msac-guidelines---clean-version---28-august-2020-3.​pdf [accessed 2024-03-28]
• Haute Autorité de Santé. Methodological choices for the clinical development of medical devices. Paris. Haute Autorité de Santé; 2013.
• Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res. Jun 29, 2017;19(6):e232. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. Jul 08, 2015;17(7):e166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. Nov 2016;51(5):843-851. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Steventon A, Grieve R, Bardsley M. An approach to assess generalizability in comparative effectiveness research: a case study of the whole systems demonstrator cluster randomized trial comparing telehealth with usual care for patients with chronic health conditions. Med Decis Making. May 18, 2015;35(8):1023-1036. [ CrossRef ]
• D2.1 Knowledge Tool 1. Health apps assessment frameworks. European mHealth Hub. 2020. URL: https://mhealth-hub.org/download/d2-1-knowledge-tool-1-health-apps-assessment-frameworks [accessed 2024-03-28]
• Moshi MR, Tooher R, Merlin T. Suitability of current evaluation frameworks for use in the health technology assessment of mobile medical applications: a systematic review. Int J Technol Assess Health Care. Sep 11, 2018;34(5):464-475. [ CrossRef ]
• Stensgaard T, Sørensen T. Telemedicine in Greenland — the creation of an evaluation plan. J Telemed Telecare. Jun 22, 2016;7(1_suppl):37-38. [ CrossRef ]
• HL7 Consumer Mobile Health Application Functional Framework (cMHAFF). HL7 International. 2018. URL: http://www.hl7.org/implement/standards/product_brief.cfm?product_id=476 [accessed 2024-03-28]
• mHealth. Kantonen K-uKvBu. 2017. URL: https://www.e-health-suisse.ch/gemeinschaften-umsetzung/ehealth-aktivitaeten/mhealth.html [accessed 2023-07-18]
• mHealthBelgium. mHealthBELGIUM. URL: https://mhealthbelgium.be/ [accessed 2024-03-28]
• Mookherji S, Mehl G, Kaonga N, Mechael P. Unmet need: improving mhealth evaluation rigor to build the evidence base. J Health Commun. Jun 04, 2015;20(10):1224-1229. [ CrossRef ] [ Medline ]
• MySNS Selecção. Serviços Partilhados do Minisério da Saúde. URL: https://mysns.min-saude.pt/mysns-seleccao-processo-de-avaliacao/ [accessed 2023-07-18]
• Nielsen S, Rimpiläinen S. Report on international practice on digital apps. Glasgow. Digital Health & Care Institute; 2018.
• Accreditation of Digital Health solutions is a fundamental foundation for their safe adoption, equipping healthcare providers and practitioners with access to health apps assured to your standards Internet. Organisation for the Review of Care and Health Applications (ORCHA). URL: https://orchahealth.com/services/ [accessed 2023-07-18]
• mHealth. TIC Salut Social. 2022. URL: https://ticsalutsocial.cat/projecte/mhealth/ [accessed 2024-03-28]
• Wienert J, Jahnel T, Maaß L. What are digital public health interventions? First steps toward a definition and an intervention classification framework. J Med Internet Res. Jun 28, 2022;24(6):e31921. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Deniz-Garcia A, Fabelo H, Rodriguez-Almeida AJ, Zamora-Zamorano G, Castro-Fernandez M, Alberiche Ruano MDP, et al. WARIFA Consortium. Quality, usability, and effectiveness of mhealth apps and the role of artificial intelligence: current scenario and challenges. J Med Internet Res. May 04, 2023;25:e44030. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World Health Organization (WHO). Classification of digital interventions, services and applications in health: a shared language to describe the uses of digital technology for health, 2nd ed. Geneva. WHO; Oct 24, 2023.
• European Parliament, Council of the European Union. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. EUR-Lex. 2017. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32017R0745 [accessed 2024-03-28]
• European Parliament, Council of the European Union. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU. EUR-Lex. 2017. URL: https://eur-lex.europa.eu/eli/reg/2017/746/oj [accessed 2024-03-28]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo R. Marco de evaluación de tecnologías sanitarias: adaptación para la evaluación de tecnologías de salud digital. Madrid, Barcelona. Ministerio de Sanidad, Agència de Qualitat i Avaluació Sanitàries de Catalunya; 2023.
• Benedetto V, Filipe L, Harris C, Spencer J, Hickson C, Clegg A. Analytical frameworks and outcome measures in economic evaluations of digital health interventions: a methodological systematic review. Med Decis Making. Oct 19, 2022;43(1):125-138. [ CrossRef ]

Abbreviations

Edited by T Leung; submitted 03.05.23; peer-reviewed by R Gorantla, KL Mauco, M Aymerich, J Haverinen, M Behzadifar; comments to author 10.11.23; revised version received 01.12.23; accepted 20.02.24; published 10.04.24.

©Joan Segur-Ferrer, Carolina Moltó-Puigmartí, Roland Pastells-Peiró, Rosa Maria Vivanco-Hidalgo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Innovative PaddleSat Concept for Thin Satellite Construction Honored

Oct 10, 2023 —.

Student researchers Vaibhav Ghosale (middle) and Grishma Kalepu (right) being presented the WiSEE Best Paper Award from Juan Fraire, the conference’s technical program committee chair.

A team of researchers from the Georgia Tech School of Electrical and Computer Engineering (ECE) has won the Best Paper Award at the 2023 IEEE Wireless in Space and Extreme Environments (WiSEE) international conference.

The team is comprised of Professor Greg Durgin and graduate students Vaibhav Bhosale, Jonathan Dolan, Grishma Kalepu, and Deeksha Manjunath. Their award-winning paper, “PaddleSats: Attitude Control and Station-Keeping for Ultra-Low Density SSP Satellites,” was selected from a field of 37 international papers.

The research presents the authors’ new PaddleSat concept in which satellites are constructed from uniformly thin surfaces, using deflections in their solar panels and subsequent changes in solar pressure-induced momentum to perform station-keeping operations. Such satellites have low launch costs and very long lifetimes in space, as there is no longer the need to carry station-keeping fuel.

PaddleSats also mitigate space debris concerns as the uniformly thin satellites will—without intervention from their controller—gradually descend from orbit (either in fragments or as a whole) due to the influence of solar pressure on the spacecraft. The PaddleSat concept is crucial for developing space solar power satellites for green energy or even low-cost communication satellites that do not contribute long-term orbital debris.

The four students who co-authored the paper began investigating the concept in 2022 as an “ECE 6390 Satellite Communications and Navigation Systems” course project. They continued to refine their work after the course, which led to this original, award-winning concept paper.

The IEEE WiSEE international conference series has been a home for the top-tier research in wireless-related systems in space for the last 11 years. The researchers were honored at this year’s conference held in Aveiro, Portugal from September 6-8.

Top photo caption: Student researchers Vaibhav Ghosale (middle) and Grishma Kalepu (right) being presented the WiSEE Best Paper Award from Juan Fraire, the conference’s technical program committee chair.

Fifth Generation (5G) Wireless Technology “Revolution in Telecommunication”

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Social justice
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

A new way to detect radiation involving cheap ceramics

Press contact :.

Previous image Next image

The radiation detectors used today for applications like inspecting cargo ships for smuggled nuclear materials are expensive and cannot operate in harsh environments, among other disadvantages. Now, in work funded largely by the U.S. Department of Homeland Security with early support from the U.S. Department of Energy, MIT engineers have demonstrated a fundamentally new way to detect radiation that could allow much cheaper detectors and a plethora of new applications.

They are working with Radiation Monitoring Devices , a company in Watertown, Massachusetts, to transfer the research as quickly as possible into detector products.

In a 2022 paper in Nature Materials , many of the same engineers reported for the first time how ultraviolet light can significantly improve the performance of fuel cells and other devices based on the movement of charged atoms, rather than those atoms’ constituent electrons.

In the current work, published recently in Advanced Materials , the team shows that the same concept can be extended to a new application: the detection of gamma rays emitted by the radioactive decay of nuclear materials.

“Our approach involves materials and mechanisms very different than those in presently used detectors, with potentially enormous benefits in terms of reduced cost, ability to operate under harsh conditions, and simplified processing,” says Harry L. Tuller, the R.P. Simmons Professor of Ceramics and Electronic Materials in MIT’s Department of Materials Science and Engineering (DMSE).

Tuller leads the work with key collaborators Jennifer L. M. Rupp, a former associate professor of materials science and engineering at MIT who is now a professor of electrochemical materials at Technical University Munich in Germany, and Ju Li, the Battelle Energy Alliance Professor in Nuclear Engineering and a professor of materials science and engineering. All are also affiliated with MIT’s Materials Research Laboratory

“After learning the Nature Materials work, I realized the same underlying principle should work for gamma-ray detection — in fact, may work even better than [UV] light because gamma rays are more penetrating — and proposed some experiments to Harry and Jennifer,” says Li.

Says Rupp, “Employing shorter-range gamma rays enable [us] to extend the opto-ionic to a radio-ionic effect by modulating ionic carriers and defects at material interfaces by photogenerated electronic ones.”

Other authors of the Advanced Materials paper are first author Thomas Defferriere, a DMSE postdoc, and Ahmed Sami Helal, a postdoc in MIT’s Department of Nuclear Science and Engineering.

Modifying barriers

Charge can be carried through a material in different ways. We are most familiar with the charge that is carried by the electrons that help make up an atom. Common applications include solar cells. But there are many devices — like fuel cells and lithium batteries — that depend on the motion of the charged atoms, or ions, themselves rather than just their electrons.

The materials behind applications based on the movement of ions, known as solid electrolytes, are ceramics. Ceramics, in turn, are composed of tiny crystallite grains that are compacted and fired at high temperatures to form a dense structure. The problem is that ions traveling through the material are often stymied at the boundaries between the grains.

In their 2022 paper, the MIT team showed that ultraviolet (UV) light shone on a solid electrolyte essentially causes electronic perturbations at the grain boundaries that ultimately lower the barrier that ions encounter at those boundaries. The result: “We were able to enhance the flow of the ions by a factor of three,” says Tuller, making for a much more efficient system.

Vast potential

At the time, the team was excited about the potential of applying what they’d found to different systems. In the 2022 work, the team used UV light, which is quickly absorbed very near the surface of a material. As a result, that specific technique is only effective in thin films of materials. (Fortunately, many applications of solid electrolytes involve thin films.)

Light can be thought of as particles — photons — with different wavelengths and energies. These range from very low-energy radio waves to the very high-energy gamma rays emitted by the radioactive decay of nuclear materials. Visible light — and UV light — are of intermediate energies, and fit between the two extremes.

The MIT technique reported in 2022 worked with UV light. Would it work with other wavelengths of light, potentially opening up new applications? Yes, the team found. In the current paper they show that gamma rays also modify the grain boundaries resulting in a faster flow of ions that, in turn, can be easily detected. And because the high-energy gamma rays penetrate much more deeply than UV light, “this extends the work to inexpensive bulk ceramics in addition to thin films,” says Tuller. It also allows a new application: an alternative approach to detecting nuclear materials.

Today’s state-of-the-art radiation detectors depend on a completely different mechanism than the one identified in the MIT work. They rely on signals derived from electrons and their counterparts, holes, rather than ions. But these electronic charge carriers must move comparatively great distances to the electrodes that “capture” them to create a signal. And along the way, they can be easily lost as they, for example, hit imperfections in a material. That’s why today’s detectors are made with extremely pure single crystals of material that allow an unimpeded path. They can be made with only certain materials and are difficult to process, making them expensive and hard to scale into large devices.

Using imperfections

In contrast, the new technique works because of the imperfections — grains — in the material. “The difference is that we rely on ionic currents being modulated at grain boundaries versus the state-of-the-art that relies on collecting electronic carriers from long distances,” Defferriere says.

Says Rupp, “It is remarkable that the bulk ‘grains’ of the ceramic materials tested revealed high stabilities of the chemistry and structure towards gamma rays, and solely the grain boundary regions reacted in charge redistribution of majority and minority carriers and defects.”

Comments Li, “This radiation-ionic effect is distinct from the conventional mechanisms for radiation detection where electrons or photons are collected. Here, the ionic current is being collected.”

Igor Lubomirsky, a professor in the Department of Materials and Interfaces at the Weizmann Institute of Science, Israel, who was not involved in the current work, says, “I found the approach followed by the MIT group in utilizing polycrystalline oxygen ion conductors very fruitful given the [materials’] promise for providing reliable operation under irradiation under the harsh conditions expected in nuclear reactors where such detectors often suffer from fatigue and aging. [They also] benefit from much-reduced fabrication costs.”

As a result, the MIT engineers are hopeful that their work could result in new, less expensive detectors. For example, they envision trucks loaded with cargo from container ships driving through a structure that has detectors on both sides as they leave a port. “Ideally, you’d have either an array of detectors or a very large detector, and that’s where [today’s detectors] really don’t scale very well,” Tuller says.

Another potential application involves accessing geothermal energy, or the extreme heat below our feet that is being explored as a carbon-free alternative to fossil fuels. Ceramic sensors at the ends of drill bits could detect pockets of heat — radiation — to drill toward. Ceramics can easily withstand extreme temperatures of more than 800 degrees Fahrenheit and the extreme pressures found deep below the Earth’s surface.

The team is excited about additional applications for their work. “This was a demonstration of principle with just one material,” says Tuller, “but there are thousands of other materials good at conducting ions.”

Concludes Defferriere: “It’s the start of a journey on the development of the technology, so there’s a lot to do and a lot to discover.”

This work is currently supported by the U.S. Department of Homeland Security, Countering Weapons of Mass Destruction Office. This support does not constitute an express or implied endorsement on the part of the government. It was also funded by the U.S. Defense Threat Reduction Agency.

Share this news article on:

Related links.

• Harry Tuller
• Tuller Research Group
• Materials Research Laboratory

Related Topics

• Nuclear security and policy
• Materials science and engineering
• Nuclear science and engineering
• Department of Energy (DoE)

Related Articles

A simple way to significantly increase lifetimes of fuel cells and other devices

Harry Tuller honored for career advancing solid-state chemistry and electrochemistry

Light could boost performance of fuel cells, lithium batteries, and other devices

Previous item Next item

More MIT News

A biomedical engineer pivots from human movement to women’s health

Read full story →

MIT tops among single-campus universities in US patents granted

A crossroads for computing at MIT

Growing our donated organ supply

New AI method captures uncertainty in medical images

Improving drug development with a vast map of the immune system

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

• Share full article

There’s an Explosion of Plastic Waste. Big Companies Say ‘We’ve Got This.’

Big brands like Procter & Gamble and Nestlé say a new generation of plants will help them meet environmental goals, but the technology is struggling to deliver.

Recycled polypropylene pellets at a PureCycle Technologies plant in Ironton, Ohio. Credit... Maddie McGarvey for The New York Times

Supported by

By Hiroko Tabuchi

• Published April 5, 2024 Updated April 8, 2024

By 2025, Nestle promises not to use any plastic in its products that isn’t recyclable. By that same year, L’Oreal says all of its packaging will be “refillable, reusable, recyclable or compostable.”

And by 2030, Procter & Gamble pledges that it will halve its use of virgin plastic resin made from petroleum.

To get there, these companies and others are promoting a new generation of recycling plants, called “advanced” or “chemical” recycling, that promise to recycle many more products than can be recycled today.

So far, advanced recycling is struggling to deliver on its promise. Nevertheless, the new technology is being hailed by the plastics industry as a solution to an exploding global waste problem.

The traditional approach to recycling is to simply grind up and melt plastic waste. The new, advanced-recycling operators say they can break down the plastic much further, into more basic molecular building blocks, and transform it into new plastic.

PureCycle Technologies, a company that features prominently in Nestlé, L’Oréal, and Procter & Gamble’s plastics commitments, runs one such facility, a $500 million plant in Ironton, Ohio. The plant was originally to start operating in 2020 , with the capacity to process as much as 182 tons of discarded polypropylene, a hard-to-recycle plastic used widely in single-use cups, yogurt tubs, coffee pods and clothing fibers, every day.

But PureCycle’s recent months have instead been filled with setbacks: technical issues at the plant, shareholder lawsuits, questions over the technology and a startling report from contrarian investors who make money when a stock price falls. They said that they had flown a drone over the facility that showed that the plant was far from being able to make much new plastic.

PureCycle, based in Orlando, Fla., said it remained on track. “We’re ramping up production,” its chief executive, Dustin Olson, said during a recent tour of the plant, a constellation of pipes, storage tanks and cooling towers in Ironton, near the Ohio River. “We believe in this technology. We’ve seen it work,” he said. “We’re making leaps and bounds.”

Nestlé, Procter & Gamble and L’Oréal have also expressed confidence in PureCycle. L’Oréal said PureCycle was one of many partners developing a range of recycling technologies. P.&G. said it hoped to use the recycled plastic for “numerous packaging applications as they scale up production.” Nestlé didn’t respond to requests for comment, but has said it is collaborating with PureCycle on “groundbreaking recycling technologies.”

PureCycle’s woes are emblematic of broad trouble faced by a new generation of recycling plants that have struggled to keep up with the growing tide of global plastic production, which scientists say could almost quadruple by midcentury .

A chemical-recycling facility in Tigard, Ore., a joint venture between Agilyx and Americas Styrenics, is in the process of shutting down after millions of dollars in losses. A plant in Ashley, Ind., that had aimed to recycle 100,000 tons of plastic a year by 2021 had processed only 2,000 tons in total as of late 2023, after fires, oil spills and worker safety complaints.

At the same time, many of the new generation of recycling facilities are turning plastic into fuel, something the Environmental Protection Agency doesn’t consider to be recycling, though industry groups say some of that fuel can be turned into new plastic .

Overall, the advanced recycling plants are struggling to make a dent in the roughly 36 million tons of plastic Americans discard each year, which is more than any other country. Even if the 10 remaining chemical-recycling plants in America were to operate at full capacity, they would together process some 456,000 tons of plastic waste, according to a recent tally by Beyond Plastics , a nonprofit group that advocates stricter controls on plastics production. That’s perhaps enough to raise the plastic recycling rate — which has languished below 10 percent for decades — by a single percentage point.

For households, that has meant that much of the plastic they put out for recycling doesn’t get recycled at all, but ends up in landfills. Figuring out which plastics are recyclable and which aren’t has turned into, essentially, a guessing game . That confusion has led to a stream of non-recyclable trash contaminating the recycling process, gumming up the system.

“The industry is trying to say they have a solution,” said Terrence J. Collins, a professor of chemistry and sustainability science at Carnegie Mellon University. “It’s a non-solution.”

‘Molecular washing machine’

It was a long-awaited day last June at PureCycle’s Ironton facility: The company had just produced its first batch of what it describes as “ultra-pure” recycled polypropylene pellets.

That milestone came several years late and with more than $350 million in cost overruns. Still, the company appeared to have finally made it. “Nobody else can do this,” Jeff Kramer, the plant manager, told a local news crew .

PureCycle had done it by licensing a game-changing method — developed by Procter & Gamble researchers in the mid-2010s, but unproven at scale — that uses solvent to dissolve and purify the plastic to make it new again. “It’s like a molecular washing machine,” Mr. Olson said.

There’s a reason Procter & Gamble, Nestlé and L’Oréal, some of the world’s biggest users of plastic, are excited about the technology. Many of their products are made from polypropylene, a plastic that they transform into a plethora of products using dyes and fillers. P.&G. has said it uses more polypropylene than any other plastic, more than a half-million tons a year.

But those additives make recycling polypropylene more difficult.

The E.P.A. estimates that 2.7 percent of polypropylene packaging is reprocessed. But PureCycle was promising to take any polypropylene — disposable beer cups, car bumpers, even campaign signs — and remove the colors, odors, and contaminants to transform it into new plastic.

Soon after the June milestone, trouble hit.

On Sept. 13, PureCycle disclosed that its plant had suffered a power failure the previous month that had halted operations and caused a vital seal to fail. That meant the company would be unable to meet key milestones, it told lenders.

Then in November, Bleecker Street Research — a New York-based short-seller, an investment strategy that involves betting that a company’s stock price will fall — published a report asserting that the white pellets that had rolled off PureCycle’s line in June weren’t recycled from plastic waste. The short-sellers instead claimed that the company had simply run virgin polypropylene through the system as part of a demonstration run.

Mr. Olson said PureCycle hadn’t used consumer waste in the June 2023 run, but it hadn’t used virgin plastic, either. Instead it had used scrap known as “post industrial,” which is what’s left over from the manufacturing process and would otherwise go to a landfill, he said.

Bleecker Street also said it had flown heat-sensing drones over the facility and said it found few signs of commercial-scale activity. The firm also raised questions about the solvent PureCycle was using to break down the plastic, calling it “a nightmare concoction” that was difficult to manage.

PureCycle is now being sued by other investors who accuse the company of making false statements and misleading investors about its setbacks.

Mr. Olson declined to describe the solvent. Regulatory filings reviewed by The New York Times indicate that it is butane, a highly flammable gas, stored under pressure. The company’s filing described the risks of explosion, citing a “worst case scenario” that could cause second-degree burns a half-mile away, and said that to mitigate the risk the plant was equipped with sprinklers, gas detectors and alarms.

Chasing the ‘circular economy’

It isn’t unusual, of course, for any new technology or facility to experience hiccups. The plastics industry says these projects, once they get going, will bring the world closer to a “circular” economy, where things are reused again and again.

Plastics-industry lobbying groups are promoting chemical recycling. At a hearing in New York late last year, industry lobbyists pointed to the promise of advanced recycling in opposing a packaging-reduction bill that would eventually mandate a 50 percent reduction in plastic packaging. And at negotiations for a global plastics treaty , lobby groups are urging nations to consider expanding chemical recycling instead of taking steps like restricting plastic production or banning plastic bags.

A spokeswoman for the American Chemistry Council, which represents plastics makers as well as oil and gas companies that produce the building blocks of plastic, said that chemical recycling potentially “complements mechanical recycling, taking the harder-to-recycle plastics that mechanical often cannot.”

Environmental groups say the companies are using a timeworn strategy of promoting recycling as a way to justify selling more plastic, even though the new recycling technology isn’t ready for prime time. Meanwhile, they say, plastic waste chokes rivers and streams, piles up in landfills or is exported .

“These large consumer brand companies, they’re out over their skis,” said Judith Enck, the president of Beyond Plastics and a former regional E.P.A. administrator. “Look behind the curtain, and these facilities aren’t operating at scale, and they aren’t environmentally sustainable,” she said.

The better solution, she said, would be, “We need to make less plastic.”

Touring the plant

Mr. Olson recently strolled through a cavernous warehouse at PureCycle’s Ironton site, built at a former Dow Chemical plant. Since January, he said, PureCycle has been processing mainly consumer plastic waste and has produced about 1.3 million pounds of recycled polypropylene, or about 1 percent of its annual production target.

“This is a bag that would hold dog food,” he said, pointing to a bale of woven plastic bags. “And these are fruit carts that you’d see in street markets. We can recycle all of that, which is pretty cool.”

The plant was dealing with a faulty valve discovered the day before, so no pellets were rolling off the line. Mr. Olson pulled out a cellphone to show a photo of a valve with a dark line ringing its interior. “It’s not supposed to look like that,” he said.

The company later sent video of Mr. Olson next to white pellets once again streaming out of its production line.

PureCycle says every kilogram of polypropylene it recycles emits about 1.54 kilograms of planet-warming carbon dioxide. That’s on par with a commonly used industry measure of emissions for virgin polypropylene. PureCycle said that it was improving on that measure.

Nestlé, L’Oréal and Procter & Gamble continue to say they’re optimistic about the technology. In November, Nestlé said it had invested in a British company that would more easily separate out polypropylene from other plastic waste.

It was “just one of the many steps we are taking on our journey to ensure our packaging doesn’t end up as waste,” the company said.

Hiroko Tabuchi covers the intersection of business and climate for The Times. She has been a journalist for more than 20 years in Tokyo and New York. More about Hiroko Tabuchi

Learn More About Climate Change

Have questions about climate change? Our F.A.Q. will tackle your climate questions, big and small .

“Buying Time,” a new series from The New York Times, looks at the risky ways  humans are starting to manipulate nature  to fight climate change.

Big brands like Procter & Gamble and Nestlé say a new generation of recycling plants will help them meet environmental goals, but the technology is struggling to deliver .

The Italian energy giant Eni sees future profits from collecting carbon dioxide and pumping it  into natural gas fields that have been exhausted.

New satellite-based research reveals how land along the East Coast is slumping into the ocean, compounding the danger from global sea level rise . A major culprit: the overpumping of groundwater.

Did you know the ♻ symbol doesn’t mean something is actually recyclable ? Read on about how we got here, and what can be done.

Advertisement