This project was partially funded with a grant from the ERC (OuterRetina) and ISF. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We thank David Buzaglo, Cheni Hermon, Liron Ben Ari and Adi Ben Ari for their assistance with designing the original version of the protocol.

Healthy participants (n = 18, aged 18-32 years, both males and females) were recruited for this study, which was approved by the Tel Aviv University Ethics Review Board (Approval No. 0004877-3). The protocol adheres to the board&#39;s guidelines for research involving human participants. Informed consent was obtained from all participants in accordance with institutional requirements.
1. Experimenter briefing
<ol>
	<li>Ask participants to perform a series of 14 distinct finger gestures (see Figure 1) and repeat each gesture 7x in a random sequence. Ask them to maintain each gesture firmly for 5 s, followed by a 3 s rest period. The total duration for each session is 13:04 min.</li>
	<li>A large image of the gesture displayed on a computer screen is accompanied by a countdown timer to indicate gesture performance. During the rest period, ask the participant to look at the small image of the upcoming gesture shown, along with a timer indicating the remaining rest time. Two distinct beep sounds signal the start and end of each gesture, helping participants prepare for the next gesture.</li>
	<li>Ask each participant to execute the procedure in four different positions, similar to previously presented22: 
	Position 1: Participant standing. Hand down, straight, and relaxed. 
	Position 2: Participant sitting in the armchair. Hand extended forward at 90&#176;, palm relaxed (a support device may be used). 
	Position 3: Hand folded upwards (with an elbow resting on the armchair), palm relaxed. 
	Position 4: Participant chooses one of the previous positions and may move the hand freely within the camera&#39;s detection range, monitored in real-time on a PC screen (see step 1.4 for more details).</li>
	<li>For each session, make the participant wear an electromyography device on the arm and position a hand-tracking camera towards them. Ask the participants to ensure that their palms always face the camera. The hand-tracking software is displayed on a separate screen so that both the participant and the conductor can verify that the hand is correctly recognized.</li>
	<li>For each position, adjust the hand-tracking camera&#39;s position and angle to ensure accurate hand recognition. Additionally, assess the quality of the signals from the electrodes using the spectrogram script.</li>
</ol>
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/67766/67766fig01.jpg" /> 
Figure 1: Schematic representation of the data collection process. The subject is equipped with a soft electrode array placed on the forearm (3), which captures high-resolution surface electromyography (sEMG) signals during gesture performance. The subject performs 14 different finger gestures presented in random order on a computer display (4). The EMG data is streamed wirelessly to a personal computer (PC) from the data acquisition unit (DAU; 1). Simultaneously, hand kinematic data (HKD) representing finger joint angles is captured using a hand-tracking camera (2). <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/67766/67766fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
2. Setting up the data acquisition units
<ol>
	<li>Open Github repository at&#160;https://github.com/NeuroEngLabTAU/Fingers_Gestures_Recognition.git and follow the detailed instructions in the Installation section. Locate the primary Python file data_collection.py in the folder finger_pose_estimation/data_acquisition. Use this to run the experiment, use the script spectrogram.py to assess EMG signal quality before the experiment begins, and the script data_analysis.py for signal filtering and segmentation.</li>
	<li>Ensure that the EMG Data Acquisition Unit (DAU) is fully charged before each session and turn it on.</li>
	<li>Connect the DAU to the PC through Bluetooth using the dedicated application. Set the Bluetooth communication rate to 500 samples per second (S/s).</li>
	<li>Install and open the hand-tracking camera&#39;s software on the PC. Connect the hand-tracking camera to the PC using a cable.</li>
	<li>Use one screen to always display hand-tracking camera software. This way, the conductor and the participant will be able to ensure that the camera recognizes the hand correctly during the experiment.</li>
</ol>
3. Participant preparation
<ol>
	<li>Introduction and consent
	<ol>
		<li>Briefly explain the study&#39;s relevance and the experimental procedure to the participant. Obtain informed consent following institutional guidelines for research involving humans.</li>
	</ol>
	</li>
	<li>Electrode placement
	<ol>
		<li>Instruct the participant to flex their right hand by forming a strong fist. While the participant flexes, palpate the forearm by gently pressing along the muscle to identify the spot where muscle activation is most prominent. This location is easily identifiable by feeling the area where the muscle bulges during contraction.</li>
		<li>Optional: Prepare the identified skin area by cleaning with an alcohol fiber-less cloth, prep gel, or water and soap. Allow the area to air dry. Avoid excessive cleaning with alcohol, as it may dry the skin. This step is optional; see the discussion section.</li>
		<li>Peel off the white protective layer from the EMG electrode array and carefully attach the electrodes to the identified forearm area as determined in step 3.2.1. (see Figure 1). Ensure the adhesive tape is closer to the palm. Secure the electrode array to the skin by gently tapping.</li>
		<li>Once the electrode array is attached to the skin, peel off the transparent support layer.</li>
		<li>Insert the electrode array connector card into the DAU&#39;s connector socket. Attach the DAU to the adhesive tape next to the electrodes.</li>
		<li>Run custom Python spectrogram script Spectrogram.py to verify real-time signal quality. A window will appear displaying raw data (on the left) and frequency domain (on the right) for all electrodes (see Supplementary Figure 1 for reference).
		<ol>
			<li>Verify that all electrodes are detected and function properly and that the signal is clean from excessive noise and 50 Hz noise.</li>
			<li>If needed, reduce 50 Hz noise by moving away from electronic devices that may cause interference and unplugging unnecessary devices from the power. Allow time for the signal to stabilize.</li>
			<li>Verify EMG signal capture: instruct participant to place an elbow on the armchair and move fingers, then relax. Ensure that a clear EMG signal is displayed followed by static baseline noise.</li>
			<li>Close the script once signal verification is complete.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Gesture and hand position review
	<ol>
		<li>Open Images folder by clicking on Finger_pose_estimation &#62; Data_acquisition. Review the gesture images with the participants.</li>
		<li>Ensure they understand each movement and can perform them accurately. Explain the four hand positions clearly to the participant.</li>
		<li>Instruct the participant on how to hold the hand before each session, ensuring proper posture and positioning.</li>
	</ol>
	</li>
	<li>Participant and camera positioning
	<ol>
		<li>For hand position 1, instruct the participant to stand straight about 1 m away from the table. Instruct the participant to hold the right hand down, straight and relaxed, with the palm facing the hand-tracking camera. Fix the hand-tracking camera on the table with a selfie stick and direct it to face the participant&#39;s hand.</li>
		<li>For hand position 2, instruct the participant to sit comfortably in an armchair positioned 40-70 cm from the monitors. Instruct the participant to extend the right hand forward at 90&#176; with a relaxed palm facing the hand-tracking camera. Use a support device, if needed, to hold the hand stable. Place the hand-tracking camera on the table facing up. 
		NOTE: As the participant is requested to remain in a fixed posture, it is important to find a comfortable position they can maintain throughout the session.</li>
		<li>For hand position 3, instruct the participant to sit as described in step 3.4.2. Instruct the participant to fold the hand upwards while resting the elbow on the armchair. The palm should be relaxed, and the participant should face the hand-tracking camera. Fix the hand-tracking camera on the table facing the participant&#39;s hand (use a selfie stick if necessary). Ensure the participant&#39;s position is optimal for both viewing the screens and being within the camera&#39;s field of view.</li>
		<li>Continuously monitor the screen displaying hand-tracking data to ensure the camera detects the hand and fingers throughout the experiment. Optional: verify EMG signal quality (step 3.2.6.) in each hand position before starting the experiment.</li>
	</ol>
	</li>
</ol>
4. Data collection
<ol>
	<li>Running the experiment
	<ol>
		<li>Open Python and load data_collection.py. Verify the parameters num_repetition, gesture_duration, rest_duration are set as desired.
		<ol>
			<li>num_repetition: Define the number of times each gesture image is shown. For this experiment, set it to 7, meaning each image is shown 7 times. gesture_duration: Specify the duration (in s) for which the participant performs the hand gesture. For this experiment, set it to 5&#160;s, determining how long each gesture image is displayed.&#160;Rest_duration: Specify the duration (in s) for which the participant relaxes their palm between gestures. For this experiment, set it to 3&#160;s.</li>
		</ol>
		</li>
		<li>Adjust the hand-tracking camera position and angle to the participant&#39;s hand position.</li>
		<li>Run the data_collection.py script. A window will appear to enter the participant&#39;s details (serial number, age, sex, session number, and hand position). Complete this information and press OK&#160;to start the experiment automatically.</li>
	</ol>
	</li>
	<li>Data collection
	<ol>
		<li>For each session, record EMG and hand-tracking data which are automatically saved. Repeat the experiment 4x for each participant, once per hand position.</li>
	</ol>
	</li>
</ol>
5. End of experiment and post-experiment data handling
<ol>
	<li>As the experiment is completed, the data are automatically saved. Ensure data is saved in a folder labeled with the participant&#39;s serial number. Each session is stored in a subfolder named S# (e.g., S1), with four subfolders for each hand position P# (P1, P2, P3, and P4). The folder size for a single session is approximately 160 MB.</li>
	<li>If a participant completes multiple sessions, ensure all data is saved in the corresponding session folder (e.g., S1, S2).</li>
	<li>Data files 
	Ensure that each hand-position folder (P#) contains the following files: EMG data saved in an EDF file, named as follows: fpe_pos{position number}_{subject number}_S{session number}_rep0_BT; hand-tracking data saved in a CSV file, named&#160;fpe_pos{position number}_{subject number}_S{session number}_rep0_BT_full; and a log file, log.txt, containing metadata about the session.</li>
	<li>Data processing 
	&#8203;NOTE: A user may choose how to proceed with signal analysis and which tools to use. Here, we provide a script for performing signal filtering and data segmentation in Python. When using Python, ensure all dependencies (e.g., Numpy, Pandas, SciPy, MNE, Sklearn) are installed.
	<ol>
		<li>Open Python, load data_analysis.py and run the script.</li>
		<li>Request will appear in the console to provide necessary parameters for data processing: path to EMG file, path to hand kinematic data, path where the processed data will be saved, sampling rate in Hz, window duration in ms, and stride interval in ms.</li>
		<li>Following that step the script will perform the data processing.</li>
		<li>EMG signal filtering: Run the script as above. The script first filters the sEMG signal by applying a 4th-order Butterworth high-pass filter with a 20 Hz cutoff to remove non-EMG signals, then a notch filter to remove 50 Hz and 100 Hz harmonics. Additionally, script applies normalization of the EMG signal.</li>
		<li>EMG, HKD data, and instructed gesture segmentation: Run the script as above. The script applies segmentation, utilizing a rolling window technique defined by the specified window duration and stride interval. In this experiment, set them to 512 and 2 ms, respectively. The script then transforms the sEMG channel organization into a 4 x 4 spatial grid configuration while maintaining the electrode array layout. Finally, the script generates a dictionary containing metadata as a pickle file.</li>
		<li>Data cleaning and validation steps
		<ol>
			<li>Identify and exclude segments containing artifacts, noise, or inconsistent gesture labels from the dataset.</li>
			<li>Ensure segment completeness and temporal continuity across windows to maintain data reliability.</li>
			<li>Cross-check gesture data against the HKD for consistency. Remove windows displaying gesture patterns that deviate from HKD session standards.</li>
			<li>Detect and discard outlier segments that fail to conform to the expected kinematic patterns for the session.</li>
			<li>Perform further data analysis using advanced algorithms. These are not provided in the current protocol.</li>
		</ol>
		</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Fang, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Simultaneous+sEMG+recognition+of+gestures+and+force+levels+for+interaction+with+prosthetic+hand.">Simultaneous sEMG recognition of gestures and force levels for interaction with prosthetic hand.</a> IEEE Trans Neural Sys Rehabilitation Eng. 30, 2426-2436 (2022).</li><li>Yadav, D., Veer, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Recent+trends+and+challenges+of+surface+electromyography+in+prosthetic+applications.">Recent trends and challenges of surface electromyography in prosthetic applications.</a> Biomed Eng Lett. 13 (3), 353-373 (2023).</li><li>Sapsanis, C., Georgoulas, G., Tzes, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=EMG+based+classification+of+basic+hand+movements+based+on+time-frequency+features.">EMG based classification of basic hand movements based on time-frequency features.</a> , 716-722 (2013).</li><li>Qaisar, S. M., Lopez, A., Dallet, D., Ferrero, F. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=sEMG+signal+based+hand+gesture+recognition+by+using+selective+subbands+coefficients+and+machine+learning.">sEMG signal based hand gesture recognition by using selective subbands coefficients and machine learning.</a> , 1-6 (2022).</li><li>Zhang, X., Zhou, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=High-density+myoelectric+pattern+recognition+toward+improved+stroke+rehabilitation.">High-density myoelectric pattern recognition toward improved stroke rehabilitation.</a> IEEE Trans Biomed Eng. 59 (6), 1649-1657 (2012).</li><li>Guo, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Empowering+hand+rehabilitation+with+AI-powered+gesture+recognition:+a+study+of+an+sEMG-based+system.">Empowering hand rehabilitation with AI-powered gesture recognition: a study of an sEMG-based system.</a> Bioengineering. 10 (5), 557 (2023).</li><li>Sun, T., Hu, Q., Libby, J., Atashzar, S. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+heterogeneous+dilation+of+LSTM+for+transient-phase+gesture+prediction+through+high-density+electromyography:+towards+application+in+neurorobotics.">Deep heterogeneous dilation of LSTM for transient-phase gesture prediction through high-density electromyography: towards application in neurorobotics.</a> IEEE Robot Autom Lett. 7 (2), 2851-2858 (2022).</li><li>Atzori, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Characterization+of+a+benchmark+database+for+myoelectric+movement+classification.">Characterization of a benchmark database for myoelectric movement classification.</a> IEEE Trans Neural Sys Rehabilitat Eng. 23 (1), 73-83 (2015).</li><li>Amma, C., Krings, T., Schultz, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Advancing+muscle-computer+interfaces+with+high-density+electromyography.">Advancing muscle-computer interfaces with high-density electromyography.</a> , 929-938 (2015).</li><li>Geng, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Gesture+recognition+by+instantaneous+surface+EMG+images.">Gesture recognition by instantaneous surface EMG images.</a> Sci Rep. 6 (1), 36571 (2016).</li><li>Wei, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multi-stream+convolutional+neural+network+for+sEMG-based+gesture+recognition+in+muscle-computer+interface.">A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface.</a> Pattern Recognit Lett. 119, 131-138 (2019).</li><li>Padhy, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+tensor-based+approach+using+multilinear+SVD+for+hand+gesture+recognition+from+sEMG+signals.">A tensor-based approach using multilinear SVD for hand gesture recognition from sEMG signals.</a> IEEE Sens J. 21 (5), 6634-6642 (2021).</li><li>Moin, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+wearable+biosensing+system+with+in-sensor+adaptive+machine+learning+for+hand+gesture+recognition.">A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition.</a> Nat Electron. 4 (1), 54-63 (2021).</li><li>C&#244;t&#233;-Allard, U., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+learning+for+electromyographic+hand+gesture+signal+classification+using+transfer+learning.">Deep learning for electromyographic hand gesture signal classification using transfer learning.</a> IEEE Trans Neural Sys Rehabilita Eng. 27 (4), 760-771 (2019).</li><li>Liu, Y., Zhang, S., Gowda, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=NeuroPose:+3D+hand+pose+tracking+using+EMG+wearables.">NeuroPose: 3D hand pose tracking using EMG wearables.</a> , 1471-1482 (2021).</li><li>Dere, M. D., Lee, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+novel+approach+to+surface+EMG-based+gesture+classification+using+a+vision+transformer+integrated+with+convolutive+blind+source+separation.">A novel approach to surface EMG-based gesture classification using a vision transformer integrated with convolutive blind source separation.</a> IEEE J Biomed Health Inform. 28 (1), 181-192 (2024).</li><li>Chen, X., Li, Y., Hu, R., Zhang, X., Chen, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hand+gesture+recognition+based+on+surface+electromyography+using+convolutional+neural+network+with+transfer+learning+method.">Hand gesture recognition based on surface electromyography using convolutional neural network with transfer learning method.</a> IEEE J Biomed Health Inform. 25 (4), 1292-1304 (2021).</li><li>Lee, K. H., Min, J. Y., Byun, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Electromyogram-based+classification+of+hand+and+finger+gestures+using+artificial+neural+networks.">Electromyogram-based classification of hand and finger gestures using artificial neural networks.</a> Sensors. 22 (1), 225 (2022).</li><li>Zhou, X., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+novel+muscle-computer+interface+for+hand+gesture+recognition+using+depth+vision.">A novel muscle-computer interface for hand gesture recognition using depth vision.</a> J Ambient Intell Humaniz Comput. 11 (11), 5569-5580 (2020).</li><li>Zhang, Z., Yang, K., Qian, J., Zhang, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-time+surface+EMG+pattern+recognition+for+hand+gestures+based+on+an+artificial+neural+network.">Real-time surface EMG pattern recognition for hand gestures based on an artificial neural network.</a> Sensors. 19 (14), 3170 (2019).</li><li>Nieuwoudt, L., Fisher, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Investigation+of+real-time+control+of+finger+movements+utilizing+surface+EMG+signals.">Investigation of real-time control of finger movements utilizing surface EMG signals.</a> IEEE Sens J. 23 (18), 21989-21997 (2023).</li><li>Ben-Ari, L., Ben-Ari, A., Hermon, C., Hanein, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Finger+gesture+recognition+with+smart+skin+technology+and+deep+learning.">Finger gesture recognition with smart skin technology and deep learning.</a> Flexible Printed Electron. 8 (2), 25012 (2023).</li><li>Yang, C., Xie, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Gesture+recognition+method+based+on+computer+vision+and+surface+electromyography:+implementing+intention+recognition+of+the+healthy+side+in+the+hand+assessment+process.">Gesture recognition method based on computer vision and surface electromyography: implementing intention recognition of the healthy side in the hand assessment process.</a> , 663-668 (2024).</li><li>Lin, J., Wu, Y., Huang, T. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Modeling+the+constraints+of+human+hand+motion.">Modeling the constraints of human hand motion.</a> Proc Workshop Human Motion. , 121-126 (2000).</li><li>Arch&#233;-N&#250;&#241;ez, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bio-potential+noise+of+dry+printed+electrodes:+physiology+versus+the+skin-electrode+impedance.">Bio-potential noise of dry printed electrodes: physiology versus the skin-electrode impedance.</a> Physiol Meas. 44 (9), 95006 (2023).</li><li>Gracia-Ib&#225;&#241;ez, V., Vergara, M., Sancho-Bru, J. L., Mora, M. C., Piqueras, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Functional+range+of+motion+of+the+hand+joints+in+activities+of+the+International+Classification+of+Functioning,+Disability+and+Health.">Functional range of motion of the hand joints in activities of the International Classification of Functioning, Disability and Health.</a> J Hand Ther. 30 (3), 337-347 (2017).</li><li>Milosevic, B., Farella, F., Benatti, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Exploring+arm+posture+and+temporal+variability+in+myoelectric+hand+gesture+recognition.">Exploring arm posture and temporal variability in myoelectric hand gesture recognition.</a> , 1032-1037 (2018).</li><li>Gijsberts, A., Atzori, M., Castellini, C., M&#252;ller, H., Caputo, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Movement+error+rate+for+evaluation+of+machine+learning+methods+for+sEMG-based+hand+movement+classification.">Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification.</a> IEEE Trans Neural Sys Rehabilitation Eng. 22 (4), 735-744 (2014).</li><li>Armitano-Lago, C., Willoughby, D., Kiefer, A. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+SWOT+analysis+of+portable+and+low-cost+markerless+motion+capture+systems+to+assess+lower-limb+musculoskeletal+kinematics+in+sport.">A SWOT analysis of portable and low-cost markerless motion capture systems to assess lower-limb musculoskeletal kinematics in sport.</a> Front Sports Act Living. 3, 809898 (2022).</li></ol>

Yael Hanein declares a financial interest in X-trodes Ltd, which commercialized the screen-printed electrode technology used in this paper. The&#160;other authors have no other relevant financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

The protocol presented in this study outlines critical steps, modifications, and troubleshooting strategies aimed at enhancing hand gesture recognition through the combination of sEMG signals and HKD. It addresses key limitations and compares this approach to existing alternatives, highlighting its potential applications in various research domains. One of the most important aspects of the protocol is ensuring the correct positioning and alignment of the hand-tracking camera. Accurate gesture capture is highly dependent on the angle and distance of the camera relative to the participant&#39;s hand. Even slight deviations in camera positioning can lead to tracking inaccuracies, reducing the fidelity of the gesture data. This alignment must be carefully adjusted for each participant and hand position to ensure consistent and reliable data collection. Additionally, it is crucial that participants are well-acquainted with the protocol to prevent junk data - where gestures are either incorrectly executed or misaligned with the experimental flow. Ensuring that participants are comfortable and familiar with the gestures and the experimental setup can minimize data noise and improve the quality of the recordings.
A common challenge in this type of study is noise contamination in both sEMG and HKD. sEMG signals are particularly sensitive to factors such as muscle fatigue, motion artifacts, and environmental noise like electromagnetic interference. Pre-processing techniques, such as band-pass filtering, are essential for reducing noise and improving signal clarity. Proper electrode placement and instructing participants to maintain relaxed muscles during rest phases can further mitigate motion artifacts. Despite these precautions, some variability in sEMG signals is inevitable due to individual differences in anatomy, hand strength, and muscle activation patterns. This variability can be addressed through flexible algorithms capable of normalizing these differences across subjects and conditions.
A key factor in achieving high-quality sEMG signals is initial signal verification. Traditional protocols using gel electrodes require skin preparation, such as exfoliating or cleaning with alcohol, to improve signal clarity. However, in a previous study we showed that with dry electrodes, skin preparation may not significantly impact signal quality25. In this protocol, skin cleaning is optional and thus simplifies the process. Another skin-related issue affecting signal quality is excessive and thick arm hair. In such cases, we suggest either shaving the area or excluding the subject from the study.
One of the critical challenges in using sEMG for gesture recognition is its sensitivity to hand positioning. Even when performing the same gesture, variations in hand orientation can lead to different EMG signal patterns. To address this issue, machine learning models that can accommodate variability in hand positions are essential22. These models must be trained with data from multiple hand postures to improve robustness and generalizability. Synchronization of visual and sEMG data is another important consideration. Consistent timing of gestures is critical to avoid discrepancies between the gesture execution and the data recording. This protocol uses visual countdowns and auditory cues to help ensure accurate timing and recalibration steps are employed when necessary to correct any misalignment during data collection.
Despite its strengths, this protocol has several limitations. One major constraint is the limited field of view of the hand-tracking camera, which requires the participant&#39;s hands to remain within the camera&#39;s detection range. This restricts the analysis to a small set of movements. For outside the lab experiments a more complex video imaging will be required or the use of smart gloves. Participant fatigue also poses a challenge during longer sessions, potentially affecting gesture accuracy and muscle activation, which can degrade the quality of the sEMG data. To mitigate these effects, it may be necessary to limit the session length or introduce breaks to minimize fatigue. Additionally, powerline interference can introduce noise into the sEMG signals, particularly when the participants are close to the PC for data capture. A wireless version of the system could reduce such interference by allowing participants to be farther from the computer.
A significant methodological limitation of EMG-based finger gesture detection stems from the high inter-subject variability in sEMG signals, which requires the development of custom models for each participant. This subject-specific approach, while more accurate, limits the protocol&#39;s scalability and requires additional calibration and training time for each new user. EMG and HKD data streams show minor temporal synchronization differences due to dual process recording. These timing discrepancies have a minimal impact on the static gesture analysis since the maintained poses are temporally stable. The sustained nature of static gestures provides adequate time for both EMG and kinematic features to stabilize, unlike dynamic gestures, which require more precise synchronization.
A key advantage of this method is its flexibility in capturing gestures. Unlike other systems that require rigid setups and strict gesture parameters, this protocol accommodates dynamic and flexible hand positions19. This flexibility is especially useful in studies aimed at analyzing a broad range of motions, making it more adaptable to real-world applications. Furthermore, this protocol is cost-effective compared to more advanced motion capture and sEMG systems, which often involve complex setups29. By integrating a hand-tracking camera with semi-automated sEMG algorithms, this method provides a viable alternative for gesture recognition studies without compromising data quality. Additionally, the system&#39;s potential for real-time data processing opens possibilities for immediate feedback in applications such as neuroprosthetics and rehabilitation, where real-time responsiveness is essential. This protocol has significant implications for several fields, particularly neuroprosthetics. Accurate prediction of hand gestures from sEMG signals is crucial for controlling prosthetic limbs, and the flexibility in hand positioning offered by this method makes it an ideal candidate for real-time prosthetic devices. In rehabilitation, this protocol could be employed to monitor and enhance motor recovery in patients with hand or finger impairments. By analyzing muscle activation patterns during gesture performance, this system could be used to tailor rehabilitation exercises to individual needs, offering a personalized approach to motor recovery. For human-computer interaction (HCI), this method enables more natural gesture-based control systems, improving the intuitiveness and efficacy of user interfaces. Lastly, the protocol could be applied to ergonomic studies to assess how different hand positions and gestures influence muscle activity and fatigue, potentially leading to advancements in workplace design and user ergonomics.
To ensure consistent contraction strength across participants, future studies could implement a glove with force-sensitive resistors to measure force directly. This would allow for standardized effort across subjects, improving the reliability of EMG data. Additionally, integrating this force measurement as a label in joint kinematics would provide a more detailed representation of the muscle&#39;s internal state, potentially enriching the analysis of muscle function and movement patterns. This approach would not only enhance data consistency but also offer deeper insights into the relationship between muscle contraction and joint motion.
In conclusion, this protocol provides a novel and flexible approach to hand gesture recognition with broad applications across neuroprosthetics, rehabilitation, HCI, and ergonomics. Although the system has limitations, its flexibility, cost-effectiveness, and potential for real-time use represent substantial advancements over existing methods. These strengths make it a promising tool for further development and innovation in gesture recognition technologies.

Hand gesturing is essential in human communication, making the recognition of finger gestures a crucial area of research across fields such as human-computer interaction, advanced prosthetics1,2,3,4, and rehabilitation technologies5,6. As a result, finger gesture recognition has garnered significant attention for its potential to enhance intuitive control systems and assistive devices. Surface electromyography (sEMG) combined with deep learning algorithms is emerging as a highly promising approach for capturing and interpreting these gestures due to its ability to detect the electrical activity of muscles associated with hand movements7,8,9,10,11,12,13,14,15.
However, despite these advances, current approaches face limitations in real-world applications. Most existing systems require complex, cumbersome recording setups with numerous electrodes5,7,9,16,17 and precise positioning3,18, which are often difficult to implement outside of controlled environments. Additionally, these systems tend to focus on static hand positions13,18,19,20,21, limiting their ability to interpret dynamic, fluid gestures that occur in daily activities. The protocol aims to address these limitations by supporting dynamic gesture recognition in more natural conditions. Such methodology would enable more practical and user-friendly applications in areas like prosthetics and rehabilitation, where real-time, natural gesture interpretation is essential.
To address these challenges, developing more accurate and adaptable algorithms requires datasets that reflect natural, everyday conditions3,4. Such datasets must capture a wide range of dynamic movements, various hand positions, and large volumes of data to ensure model robustness. Furthermore, the variability between training and test datasets is crucial, allowing models to generalize across different hand postures, muscle activation patterns, and motions. Incorporating such diversity into the data will enable algorithms to perform gesture recognition more accurately in everyday, real-world applications22.
Overcoming these challenges will be essential for the future development of more practical and widely applicable gesture recognition systems. The study and protocol described here stem&#160;from the need to have a portable, user-friendly setup that can capture dynamic hand movements in natural settings. Comprehensive datasets and advanced algorithms are critical to fully unlocking the potential of sEMG and deep learning in human-computer interfaces, neuroprosthetics, and rehabilitation technologies. We expect this protocol to contribute to the field by facilitating comprehensive data collection to further enable the development of algorithm models that generalize across diverse hand positions.
A significant challenge in gesture recognition lies in the sensitivity of sEMG signals to hand positioning. While many studies focus on fixed-hand positions for gesture prediction, real-world applications demand models capable of recognizing finger movements across various hand postures. Recent approaches have addressed this by incorporating computer vision as a ground truth reference, enhancing the accuracy and flexibility of these models15,19. Additionally, hybrid models that integrate sEMG signals with visual data offer further improvements in recognition accuracy across diverse scenarios23.
In this protocol, we present a synchronized approach to data collection that enhances dynamic gesture recognition by incorporating both EMG and hand-tracking data in real-world-like conditions. Unlike traditional methods that restrict gesture performance to static positions, this protocol includes gestures performed across four distinct positions: hand down, hand up, hand straight, and hand moving. The hand-tracking camera tracks hand movements within a three-dimensional interactive zone, identifying distinct hand elements and capturing dynamic movements with high resolution. A soft electrode array of 16 electrodes placed on the forearm to record muscle activity offers stable and wireless recordings without impeding participant mobility. The synchronized data from these two sources provides a comprehensive foundation for developing advanced gesture recognition algorithms capable of operating in real-world conditions. The approach specifically addresses the limitations of current setups by facilitating free movement and stable signal recording in realistic scenarios. This advancement supports gesture recognition technologies for applications in prosthetics, rehabilitation, and interactive technologies, where intuitive control and flexibility are essential.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Adjustable Selfie Stick</td>
			<td></td>
			<td></td>
			<td>Used to position and angle the hand-tracking camera in the desired orientation for optimal data capture during the experiment.</td>
		</tr>
		<tr>
			<td>Alcohol pad</td>
			<td></td>
			<td></td>
			<td>To clean the area for electrode placement.</td>
		</tr>
		<tr>
			<td>Data acquisition unit (DAU)</td>
			<td>X-trodes Ltd.</td>
			<td>&#160;XTR-BT V1.3</td>
			<td>Realtime compatible DAU 2.0.17 or 4.0.1</td>
		</tr>
		<tr>
			<td>Finger Gestures Recognition library</td>
			<td></td>
			<td></td>
			<td>https://github.com/NeuroEngLabTAU/Fingers_Gestures_Recognition.git</td>
		</tr>
		<tr>
			<td>Leap Motion Controller 2</td>
			<td>Ultraleap</td>
			<td>129-SP4-00005-03</td>
			<td>Hand-tracking camera</td>
		</tr>
		<tr>
			<td>Long Type-C to Type-C cable</td>
			<td></td>
			<td></td>
			<td>Connection of the hand-tracking camera to the PC.</td>
		</tr>
		<tr>
			<td>PC Monitors</td>
			<td></td>
			<td></td>
			<td>One for guidelines, one for viewing the hand-tracking camera data&#160;</td>
		</tr>
		<tr>
			<td>Personal Computer (PC)</td>
			<td>Windows</td>
			<td></td>
			<td>Windows 10+; Processors: Inteli7 processor. BT receiver.</td>
		</tr>
		<tr>
			<td>Python code</td>
			<td></td>
			<td></td>
			<td>A script enabling seamless data streaming and recording up to 500 S/s when DAU is connected to PC via Bluetooth</td>
		</tr>
		<tr>
			<td>Ultraleap Camera Python API</td>
			<td>Ultraleap</td>
			<td></td>
			<td>Python API and instructions from Ultraleap&#8217;s GitHub repository (https://github.com/ultraleap/leapc-python-bindings) used to collect data from the Ultraleap unit during the experiment</td>
		</tr>
		<tr>
			<td>Ultraleap Hyperion</td>
			<td>Ultraleap</td>
			<td></td>
			<td>Tracking software</td>
		</tr>
		<tr>
			<td>XTR EXG16</td>
			<td>X-trodes Ltd.</td>
			<td>XTELC0003405RM</td>
			<td>16-channel wearable dry electrode array patch for EMG, ECG monitoring</td>
		</tr>
		<tr>
			<td>X-trodes PC App</td>
			<td>X-trodes Ltd.</td>
			<td>1.1.35.0</td>
			<td>An application for connecting X-trodes DAU to PC via BT</td>
		</tr>
	</tbody>
</table>

capturing dynamic finger gesturing with high-resolution surface electromyography and computer vision

Finger gestures are a critical element in human communication, and as such, finger gesture recognition is widely studied as a human-computer interface for state-of-the-art prosthetics and optimized rehabilitation. Surface electromyography (sEMG), in conjunction with deep learning methods, is considered a promising method in this domain. However, current methods often rely on cumbersome recording setups and the identification of static hand positions, limiting their effectiveness in real-world applications. The protocol we report here presents an advanced approach combining a wearable surface EMG and finger&#160;tracking system to capture comprehensive data during dynamic hand movements. The method records muscle activity from soft printed electrode arrays (16 electrodes) placed on the forearm as subjects perform gestures in different hand positions and during movement. Visual instructions prompt subjects to perform specific gestures while EMG and finger positions are recorded. The integration of synchronized EMG recordings and finger tracking data enables comprehensive analysis of muscle activity patterns and corresponding gestures. The reported approach demonstrates the potential of combining EMG and visual tracking technologies as an important resource for developing intuitive and responsive gesture recognition systems with applications in prosthetics, rehabilitation, and interactive technologies. This protocol aims to guide researchers and practitioners, fostering further innovation and application of gesture recognition in dynamic and real-world scenarios.

The dataset consists of two time-synchronized components: a 16-channel EMG dataset and data from a hand-tracking camera system. The 16-channel EMG data captures muscle activity by recording electrical signals from different muscles over time. The hand-tracking system provides 16 channels of data corresponding to key points on a skeletal model of the hand. While the model has 21 points, excluding the wrist, this number was reduced to 16 due to motion constraints24. The EMG and visual data were collected by running two separate processes on the same computer during recording to establish synchrony. A timestamp was used to mark the start of each process, allowing data analysis code to align muscle activity and hand movement data at the end of the recording. Timestamp annotations were saved automatically in both EDF and CSV files, marking the exact time when specific finger gestures were instructed and facilitating the alignment during data analysis. The filtered EMG signal (20 Hz 4th-order Butterworth high pass filter) is characterized by a low baseline (grey-shaded areas), which typically falls within the range of 3-9 &#181;V25. This baseline is observed when the subject&#39;s hand is stationary and the muscles are at rest. However, if muscle tone is present even in the rest position, a distinct EMG signal can be detected. Mechanical artifacts caused by movement usually manifest in the 10-20 Hz range and should be filtered out accordingly. Significantly elevated baseline values may indicate 50 Hz line interference and should be avoided during the experimental setup stage. In cases where moderate 50 Hz noise persists; a notch filter is applied. Sharp movement artifacts, which are more difficult to remove, often appear as pronounced high-amplitude spikes in the signal (see asterisk in Figure 2A). The amplitude of the EMG signal across the 16-electrode array varies, reflecting the spatial distribution of the muscle activity over the region measured. This variance provides valuable insight into the heterogeneity of muscle contraction during hand gestures.
The hand-tracking camera provides direct information of finger angles (hand kinematic data, HKD), which are expected to correlate closely with the recorded EMG signals. During gestures, finger angles in normal range26, depending on the specific gesture. When the visual path between the hand-tracking camera and the hand is unobstructed, the resulting signal is stable and accurate, as demonstrated in Figure 2. However, in instances where visual contact is lost or when the system experiences technical limitations, the HKD may become erratic, displaying jumps between incorrect values. Such outlier data should be minimized during data collection and discarded in the final analysis to maintain the integrity of the results.
The HKD is intuitive and provides a direct comparison with the actual gestures performed. It exhibits low variability between subjects and across different hand positions. In contrast, the EMG data tends to vary significantly between individuals due to anatomical differences such as hand size and muscle development27. Additionally, variability may be observed between dominant and non-dominant hands. This subject-specific variability can be addressed during offline analysis.
In Figure 2, it is evident that both the EMG and HKD are offset relative to the instructed gesture trigger. This discrepancy arises due to response time and the natural movement execution28. In regression tasks, such variability could contribute to the richness of the data, while in classification tasks, it can be managed using a generalized likelihood ratio approach, as applied in similar scenarios28.
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/67766/67766fig02.jpg" /> 
Figure 2:&#160;Representative sEMG and HKD during finger abduction. Surface electromyography (sEMG) signals and hand kinematic data (HKD) recorded during dynamic finger abduction and rest performed during hand position 1 (hand down, straight, and relaxed) by a single participant. (A) Filtered EMG signals from 16 channels as a function of time. Asterisk (*) denotes a mechanical artifact detected in the EMG recording of Channel 5. (B) HKD, showing the joint angles as a function of time. Joint angles are measured at various joints: trapeziometacarpal (TMC), metacarpophalangeal (MCP), and proximal interphalangeal (PIP). The phases of the experiment (rest and abduction) are indicated along the x-axis. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/67766/67766fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
These representative results demonstrated the utility of the synchronized EMG and HKD data in capturing hand gestures. The alignment of EMG signals with corresponding HKD allows mapping muscle activity to specific finger movements. When constructing a predictive model, researchers can use HKD as ground truth, iteratively verifying and refining EMG-based gesture predictions. This approach highlights the practical applicability of the protocol and suggests the need for further research in more natural settings.
Supplementary Figure 1: Spectrogram windows displayed during the signal verification step. The left panels show raw EMG data, while the right panels show the detected frequency domains. (A) Example of a very noisy EMG signal with strong 50 Hz and 100 Hz interference. (B) Example of the same EMG signal recording after moving the participant further away from electrical devices, resulting in a clean EMG signal with minimal interference. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/67766/S1_R1.jpg" target="_blank">Please click here to download this File.</a>

The paper presents a comprehensive protocol for simultaneously recording hand-electromyography (EMG) and visual finger tracking during natural finger gesturing. The visual data is designed to serve as the ground truth for the development of accurate EMG-based computational models for finger gesture recognition.

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

Finger gestures are a critical element in human communication, and as such, finger gesture recognition is widely studied as a human-computer interface for ...

capturing-dynamic-finger-gesturing-with-high-resolution-surface

Nanyang Technological University

Research

JoVE Journal

Bioengineering

248 Views.  Tel Aviv University. The paper presents a comprehensive protocol for simultaneously recording hand-electromyography (EMG) and visual finger tracking during natural finger gesturing. The visual data is designed to serve as the ground truth for the development of accurate EMG-based computational models for finger gesture recognition. 

Video: Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

High-resolution Fiber-optic Microendoscopy for in situ Cellular Imaging

Many biological and clinical studies require the longitudinal study and analysis of morphology and function with cellular level resolution. Traditionally, multiple experiments are run in parallel, with individual samples removed from the study at sequential time points for evaluation by light microscopy. Several intravital techniques have been developed, with confocal, multiphoton, and second harmonic microscopy all demonstrating their ability to be used for imaging in situ 1. With these systems, however, the required infrastructure is complex and expensive, involving scanning laser systems and complex light sources. Here we present a protocol for the design and assembly of a high-resolution microendoscope which can be built in a day using off-the-shelf components for under US$5,000. The platform offers flexibility in terms of image resolution, field-of-view, and operating wavelength, and we describe how these parameters can be easily modified to meet the specific needs of the end user.
We and others have explored the use of the high-resolution microendoscope (HRME) in in vitro cell culture 2-5, in excised 6 and living animal tissues 2,5, and in human tissues in vivo 2,7. Users have reported the use of several different fluorescent contrast agents, including proflavine 2-4, benzoporphyrin-derivative monoacid ring A (BPD-MA) 5, and fluoroscein 6,7, all of which have received full, or investigational approval from the FDA for use in human subjects. High-resolution microendoscopy, in the form described here, may appeal to a wide range of researchers working in the basic and clinical sciences. The technique offers an effective and economical approach which complements traditional benchtop microscopy, by enabling the user to perform high-resolution, longitudinal imaging in situ.

High-resolution Fiber-optic Microendoscopy for in situ Cellular Imaging

In many biological and clinical situations it is advantageous to study cellular processes as they evolve in their native microenvironment. Here we describe the assembly and use of a low-cost fiber-optic microscope which can provide real time imaging in cell culture, animal studies, and clinical patient studies.

Many biological and clinical studies require the longitudinal study and analysis of morphology and function with cellular level resolution.  ...

High Resolution 3D Imaging of Ex-Vivo Biological Samples by Micro CT

Non-destructive volume visualization can be achieved only by tomographic techniques, of which the most efficient is the x-ray micro computerized tomography (&mu;CT).
High resolution &mu;CT is a very versatile yet accurate (1-2 microns of resolution) technique for 3D examination of ex-vivo biological samples1, 2. As opposed to electron tomography, the &mu;CT allows the examination of up to 4 cm thick samples. This technique requires only few hours of measurement as compared to weeks in histology. In addition, &mu;CT does not rely on 2D stereologic models, thus it may complement and in some cases can even replace histological methods3, 4, which are both time consuming and destructive. Sample conditioning and positioning in &mu;CT is straightforward and does not require high vacuum or low temperatures, which may adversely affect the structure. The sample is positioned and rotated 180&deg; or 360&deg;between a microfocused x-ray source and a detector, which includes a scintillator and an accurate CCD camera, For each angle a 2D image is taken, and then the entire volume is reconstructed using one of the different available algorithms5-7. The 3D resolution increases with the decrease of the rotation step. The present video protocol shows the main steps in preparation, immobilization and positioning of the sample followed by imaging at high resolution.

High Resolution 3D Imaging of Ex-Vivo Biological Samples by Micro CT

Non-destructive volume visualization can be achieved only by tomographic techniques, of which the most efficient is the x-ray micro computerized tomography ( CT).

Non-destructive volume visualization can be achieved only by tomographic techniques, of which the most efficient is the x-ray micro computerized ...

Continuous High-resolution Microscopic Observation of Replicative Aging in Budding Yeast

We demonstrate the use of a simple microfluidic setup, in which single budding yeast cells can be tracked throughout their entire lifespan. The microfluidic chip exploits the size difference between mother and daughter cells using an array of micropads. Upon loading, cells are trapped underneath these micropads, because the distance between the micropad and cover glass is similar to the diameter of a yeast cell (3-4 &mu;m). After the loading procedure, culture medium is continuously flushed through the chip, which not only creates a constant and defined environment throughout the entire experiment, but also flushes out the emerging daughter cells, which are not retained underneath the pads due to their smaller size. The setup retains mother cells so efficiently that in a single experiment up to 50 individual cells can be monitored in a fully automated manner for 5 days or, if necessary, longer. In addition, the excellent optical properties of the chip allow high-resolution imaging of cells during the entire aging process.

We describe here the operation of a microfluidic device that allows continuous and high-resolution microscopic imaging of single budding yeast cells during their complete replicative and/or chronological lifespan.

We demonstrate the use of a simple microfluidic setup, in which single budding yeast cells can be tracked throughout their entire lifespan. The ...

High-resolution Spatiotemporal Analysis of Receptor Dynamics by Single-molecule Fluorescence Microscopy

Single-molecule microscopy is emerging as a powerful approach to analyze the behavior of signaling molecules, in particular concerning those aspect (e.g., kinetics, coexistence of different states and populations, transient interactions), which are typically hidden in ensemble measurements, such as those obtained with standard biochemical or microscopy methods. Thus, dynamic events, such as receptor-receptor interactions, can be followed in real time in a living cell with high spatiotemporal resolution. This protocol describes a method based on labeling with small and bright organic fluorophores and total internal reflection fluorescence (TIRF) microscopy to directly visualize single receptors on the surface of living cells. This approach allows one to precisely localize receptors, measure the size of receptor complexes, and capture dynamic events such as transient receptor-receptor interactions. The protocol provides a detailed description of how to perform a single-molecule experiment, including sample preparation, image acquisition and image analysis. As an example, the application of this method to analyze two G-protein-coupled receptors, i.e., &#946;2-adrenergic and &#947;-aminobutyric acid type B (GABAB) receptor, is reported. The protocol can be adapted to other membrane proteins and different cell models, transfection methods and labeling strategies.

This protocol describes how to use total internal reflection fluorescence microscopy to visualize and track single receptors on the surface of living cells and thereby analyze receptor lateral mobility, size of receptor complexes as well as to visualize transient receptor-receptor interactions. This protocol can be extended to other membrane proteins.

Single-molecule microscopy is emerging as a powerful approach to analyze the behavior of signaling molecules, in particular concerning those aspect (e.g., ...

Preparation of Mica Supported Lipid Bilayers for High Resolution Optical Microscopy Imaging

Supported lipid bilayers (SLBs) are widely used as a model for studying membrane properties (phase separation, clustering, dynamics) and its interaction with other compounds, such as drugs or peptides. However SLB characteristics differ depending on the support used. 
Commonly used techniques for SLB imaging and measurements are single molecule fluorescence microscopy, FCS and atomic force microscopy (AFM). Because most optical imaging studies are carried out on a glass support, while AFM requires an extremely flat surface (generally mica), results from these techniques cannot be compared directly, since the charge and smoothness properties of these materials strongly influence diffusion. Unfortunately, the high level of manual dexterity required for the cutting and gluing thin slices of mica to the glass slide presents a hurdle to routine use of mica for SLB preparation. Although this would be the method of choice, such prepared mica surfaces often end up being uneven (wavy) and difficult to image, especially with small working distance, high numerical aperture lenses. Here we present a simple and reproducible method for preparing thin, flat mica surfaces for lipid vesicle deposition and SLB preparation. Additionally, our custom made chamber requires only very small volumes of vesicles for SLB formation. The overall procedure results in the efficient, simple and inexpensive production of high quality lipid bilayer surfaces that are directly comparable to those used in AFM studies.

We present a method of preparing mica supported lipid bilayers for high resolution microscopy. Mica is transparent and flat on an atomic scale, but rarely used in imaging because of handling difficulties; our preparation results in even deposition of the mica sheet, and reduces the material used in bilayer preparation.

Supported lipid bilayers (SLBs) are widely used as a model for studying membrane properties (phase separation, clustering, dynamics) and its interaction ...

Flying Insect Detection and Classification with Inexpensive Sensors

An inexpensive, noninvasive system that could accurately classify flying insects would have important implications for entomological research, and allow for the development of many useful applications in vector and pest control for both medical and agricultural entomology. Given this, the last sixty years have seen many research efforts devoted to this task. To date, however, none of this research has had a lasting impact. In this work, we show that pseudo-acoustic optical sensors can produce superior data; that additional features, both intrinsic and extrinsic to the insect&#8217;s flight behavior, can be exploited to improve insect classification; that a Bayesian classification approach allows to efficiently learn classification models that are very robust to over-fitting, and a general classification framework allows to easily incorporate arbitrary number of features. We demonstrate the findings with large-scale experiments that dwarf all previous works combined, as measured by the number of insects and the number of species considered.

We proposed a system that uses inexpensive, noninvasive pseudo-acoustic optical sensors to automatically and accurately detect, count, and classify flying insects based on their flying sound.

An inexpensive, noninvasive system that could accurately classify flying insects would have important implications for entomological research, and allow ...

Microfluidic Platform with Multiplexed Electronic Detection for Spatial Tracking of Particles

Microfluidic processing of biological samples typically involves differential manipulations of suspended particles under various force fields in order to spatially fractionate the sample based on a biological property of interest. For the resultant spatial distribution to be used as the assay readout, microfluidic devices are often subjected to microscopic analysis requiring complex instrumentation with higher cost and reduced portability. To address this limitation, we have developed an integrated electronic sensing technology for multiplexed detection of particles at different locations on a microfluidic chip. Our technology, called Microfluidic CODES, combines Resistive Pulse Sensing with Code Division Multiple Access to compress 2D spatial information into a 1D electrical signal. In this paper, we present a practical demonstration of the Microfluidic CODES technology to detect and size cultured cancer cells distributed over multiple microfluidic channels. As validated by the high-speed microscopy, our technology can accurately analyze dense cell populations all electronically without the need for an external instrument. As such, the Microfluidic CODES can potentially enable low-cost integrated lab-on-a-chip devices that are well suited for the point-of-care testing of biological samples.

We demonstrate a microfluidic platform with an integrated surface electrode network that combines resistive pulse sensing (RPS) with code division multiple access (CDMA), to multiplex detection and sizing of particles in multiple microfluidic channels.

Microfluidic processing of biological samples typically involves differential manipulations of suspended particles under various force fields in order to ...

High Resolution 3D Imaging of the Human Pancreas Neuro-insular Network

Using traditional histological methods, researchers are hampered in their ability to image whole tissues or organs in large-scale 3D. Histological sections are generally limited to &#60;20 &#181;m as formalin fixed paraffin section on glass slides or &#60;500 &#181;m for free-floating fixed sections. Therefore, extensive efforts are required for serial sectioning and large-scale image reconstruction methods to recreate 3D for samples &#62;500 &#181;m using traditional methods. In addition, light scatters from macromolecules within tissues, particularly lipids, prevents imaging to a depth &#62;150 &#181;m with most confocal microscopes. To reduce light scatter and to allow for deep tissue imaging using simple confocal microscopy, various optical clearing methods have been developed that are relevant for rodent and human tissue samples fixed by immersion. Several methods are related and use protein crosslinking with acrylamide and tissue clearing with sodium dodecyl sulfate (SDS). Other optical clearing techniques used various solvents though each modification had various advantages and disadvantages. Here, an optimized passive optical clearing method is described for studies of the human pancreas innervation and specifically for interrogation of the innervation of human islets.

Here, we present a protocol to image human pancreas sections in three dimensions (3D) using optimized passive clearing methods. This manuscript demonstrates these procedures for passive optical clearing followed by multiple immunofluorescence staining to identify key elements of the autonomic and sensory neural networks innervating human islets.

Using traditional histological methods, researchers are hampered in their ability to image whole tissues or organs in large-scale 3D. Histological ...

Surface Engineering of Pancreatic Islets with a Heparinized StarPEG Nanocoating

Cell surface engineering can protect implanted cells from host immune attack. It can also reshape cellular landscape to improve graft function and survival post-transplantation. This protocol aims to achieve surface engineering of pancreatic islets using an ultrathin heparin-incorporated starPEG (Hep-PEG) nanocoating. To generate the Hep-PEG nanocoating for pancreatic islet surface engineering, heparin succinimidyl succinate (Heparin-NHS) was first synthesized by modification of its carboxylate groups using N-(3-dimethylamino propyl)-N'-ethyl carbodiimide hydrochloride (EDC) and N-hydroxysuccinimide (NHS). The Hep-PEG mixture was then formed by crosslinking of the amino end-functionalized eight-armed starPEG (starPEG-(NH2)8) and Heparin-NHS. For islet surface coating, mouse islets were isolated via collagenase digestion and gradient purification using Histopaque. Isolated islets were then treated with ice cold Hep-PEG solution for 10 min to allow covalent binding between NHS and the amine groups of islet cell membrane. Nanocoating with the Hep-PEG incurs minimal alteration to islet size and volume and heparinization of the islets with Hep-PEG may also reduce instant blood-mediated inflammatory reaction during islet transplantation. This "easy-to-adopt" approach is mild enough for surface engineering of living cells without compromising cell viability. Considering that heparin has shown binding affinity to multiple cytokines, the Hep-PEG nanocoating also provides an open platform that enables incorporation of unlimited functional biological mediators and multi-layered surfaces for living cell surface bioengineering.

This protocol aims to achieve surface engineering of pancreatic islets using a heparin-incorporated starPEG nanocoating via pseudo-bioorthogonal chemistry between the N-hydroxysuccinimide groups of the nanocoating and the amine groups of islet cell membrane.

Cell surface engineering can protect implanted cells from host immune attack. It can also reshape cellular landscape to improve graft function and ...

High-resolution Patterning Using Two Modes of Electrohydrodynamic Jet: Drop on Demand and Near-field Electrospinning

Electrohydrodynamic (EHD) jet printing has drawn attention in various fields because it can be used as a high-resolution and low-cost direct patterning tool. EHD printing uses a fluidic supplier to maintain the extruded meniscus by pushing the ink out of the nozzle tip. The electric field is then used to pull the meniscus down to the substrate to produce high-resolution patterns. Two modes of EHD printing have been used for fine patterning: continuous near-field electrospinning (NFES) and dot-based drop-on-demand (DOD) EHD printing. According to the printing modes, the requirements for the printing equipment and ink viscosity will differ. Even though two different modes can be implemented with a single EHD printer, the realization methods significantly differ in terms of ink, fluidic system, and driving voltage. Consequently, without a proper understanding of the jetting requirements and limitations, it is difficult to obtain the desired results. The purpose of this paper is to present a guideline so that inexperienced researchers can reduce the trial and error efforts to use the EHD jet for their specific research and development purposes. To demonstrate the fine-patterning implementation, we use Ag nanoparticle ink for the conductive patterning in the protocol. In addition, we also present the generalized printing guidelines that can be used for other types of ink for various fine-patterning applications.

Here, we present a protocol to produce high-resolution conductive patterns using electrohydrodynamic (EHD) jet printing. The protocol includes two modes of EHD jet printing: the continuous near-field electrospinning (NFES) and the dot-based drop-on-demand (DOD) EHD printing.

Electrohydrodynamic (EHD) jet printing has drawn attention in various fields because it can be used as a high-resolution and low-cost direct patterning ...