{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T03:48:30Z","timestamp":1782186510882,"version":"3.54.5"},"reference-count":55,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2019,12,5]],"date-time":"2019-12-05T00:00:00Z","timestamp":1575504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, a novel method of active tactile perception based on 3D neural networks and a high-resolution tactile sensor installed on a robot gripper is presented. A haptic exploratory procedure based on robotic palpation is performed to get pressure images at different grasping forces that provide information not only about the external shape of the object, but also about its internal features. The gripper consists of two underactuated fingers with a tactile sensor array in the thumb. A new representation of tactile information as 3D tactile tensors is described. During a squeeze-and-release process, the pressure images read from the tactile sensor are concatenated forming a tensor that contains information about the variation of pressure matrices along with the grasping forces. These tensors are used to feed a 3D Convolutional Neural Network (3D CNN) called 3D TactNet, which is able to classify the grasped object through active interaction. Results show that 3D CNN performs better, and provide better recognition rates with a lower number of training data.<\/jats:p>","DOI":"10.3390\/s19245356","type":"journal-article","created":{"date-parts":[[2019,12,5]],"date-time":"2019-12-05T03:16:36Z","timestamp":1575515796000},"page":"5356","update-policy":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":46,"title":["Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0003-0006-6811","authenticated-orcid":false,"given":"Francisco","family":"Pastor","sequence":"first","affiliation":[{"name":"Robotics and Mechatronics Group, University of M\u00e1laga, 29071 M\u00e1laga, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0002-0382-334X","authenticated-orcid":false,"given":"Juan M.","family":"Gandarias","sequence":"additional","affiliation":[{"name":"Robotics and Mechatronics Group, University of M\u00e1laga, 29071 M\u00e1laga, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0003-3432-3230","authenticated-orcid":false,"given":"Alfonso J.","family":"Garc\u00eda-Cerezo","sequence":"additional","affiliation":[{"name":"Robotics and Mechatronics Group, University of M\u00e1laga, 29071 M\u00e1laga, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0001-5960-3453","authenticated-orcid":false,"given":"Jes\u00fas M.","family":"G\u00f3mez-de-Gabriel","sequence":"additional","affiliation":[{"name":"Robotics and Mechatronics Group, University of M\u00e1laga, 29071 M\u00e1laga, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,5]]},"reference":[{"key":"ref_1","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, Inc."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12559-018-9568-7","article-title":"End-to-End ConvNet for Tactile Recognition Using Residual Orthogonal Tiling and Pyramid Convolution Ensemble","volume":"10","author":"Cao","year":"2018","journal-title":"Cogn. Comput."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Shibata, A., Ikegami, A., Nakauma, M., and Higashimori, M. (2017). Convolutional Neural Network based Estimation of Gel-like Food Texture by a Robotic Sensing System. Robotics, 6.","DOI":"10.3390\/robotics6040037"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gandarias, J.M., G\u00f3mez-de Gabriel, J.M., and Garc\u00eda-Cerezo, A.J. (2017). Tactile Sensing and Machine Learning for Human and Object Recognition in Disaster Scenarios. Third Iberian Robotics Conference, Springer.","DOI":"10.1007\/978-3-319-70836-2_14"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3249","DOI":"10.3390\/s110303249","article-title":"Three realizations and comparison of hardware for piezoresistive tactile sensors","volume":"11","year":"2011","journal-title":"Sensors"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"5298","DOI":"10.1109\/JSEN.2016.2550605","article-title":"Magnetic and Mechanical Modeling of a Soft Three-Axis Force Sensor","volume":"16","author":"Chathuranga","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1089\/soro.2017.0052","article-title":"The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies","volume":"5","author":"Pestell","year":"2018","journal-title":"Soft Robot."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gong, D., He, R., Yu, J., and Zuo, G. (2017). A pneumatic tactile sensor for co-operative robots. Sensors, 17.","DOI":"10.3390\/s17112592"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3910","DOI":"10.1109\/JSEN.2013.2258149","article-title":"A Flexible and Robust Large Scale Capacitive Tactile System for Robots","volume":"13","author":"Maiolino","year":"2013","journal-title":"IEEE Sens. J."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Gandarias, J.M., G\u00f3mez-de Gabriel, J.M., and Garc\u00eda-Cerezo, A.J. (2018). Enhancing Perception with Tactile Object Recognition in Adaptive Grippers for Human\u2013Robot Interaction. Sensors, 18.","DOI":"10.3390\/s18030692"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1109\/TRO.2011.2134130","article-title":"Tactile sensing for mobile manipulation","volume":"27","author":"Chitta","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3340","DOI":"10.1109\/LRA.2018.2852797","article-title":"Slip Detection With a Biomimetic Tactile Sensor","volume":"3","author":"James","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Romeo, R., Oddo, C., Carrozza, M., Guglielmelli, E., and Zollo, L. (2017). Slippage Detection with Piezoresistive Tactile Sensors. Sensors, 17.","DOI":"10.3390\/s17081844"},{"key":"ref_14","unstructured":"Gandarias, J.M., Gomez-de Gabriel, J.M., and Garcia-Cerezo, A. (November, January 29). Human and object recognition with a high-resolution tactile sensor. Proceedings of the IEEE Sensors Conference, Glasgow, UK."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Luo, S., Mou, W., Althoefer, K., and Liu, H. (2016, January 9\u201314). Iterative Closest Labeled Point for Tactile Object Shape Recognition. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759485"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yuan, Q., and Wang, J. (2017, January 21\u201323). Design and Experiment of the NAO Humanoid Robot\u2019s Plantar Tactile Sensor for Surface Classification. Proceedings of the 4th International Conference on Information Science and Control Engineering (ICISCE), Changsha, China.","DOI":"10.1109\/ICISCE.2017.197"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hoelscher, J., Peters, J., and Hermans, T. (2015, January 3\u20135). Evaluation of tactile feature extraction for interactive object recognition. Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Seoul, Korea.","DOI":"10.1109\/HUMANOIDS.2015.7363560"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.mechatronics.2017.11.002","article-title":"Robotic tactile perception of object properties: A review","volume":"48","author":"Luo","year":"2017","journal-title":"Mechatronics"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1381","DOI":"10.1109\/TNSRE.2018.2838326","article-title":"Tactile Sensor-Based Steering as a Substitute of the Attendant Joystick in Powered Wheelchairs","volume":"26","author":"Bachta","year":"2018","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Schiefer, M.A., Graczyk, E.L., Sidik, S.M., Tan, D.W., and Tyler, D.J. (2018). Artificial tactile and proprioceptive feedback improves performance and confidence on object identification tasks. PLoS ONE, 13.","DOI":"10.1371\/journal.pone.0207659"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1038\/nmat4731","article-title":"Robots with a sense of touch","volume":"15","author":"Bartolozzi","year":"2016","journal-title":"Nat. Mater."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4226","DOI":"10.1109\/JSEN.2015.2417759","article-title":"Highly Sensitive Soft Tactile Sensors for an Anthropomorphic Robotic Hand","volume":"15","author":"Jamone","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Roncone, A., Hoffmann, M., Pattacini, U., Fadiga, L., and Metta, G. (2016). Peripersonal space and margin of safety around the body: Learning visuo-tactile associations in a humanoid robot with artificial skin. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0163713"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tanaka, Y., Nagai, T., Sakaguchi, M., Fujiwara, M., and Sano, A. (2013, January 14\u201317). Tactile sensing system including bidirectionality and enhancement of haptic perception by tactile feedback to distant part. Proceedings of the IEEE World Haptics Conference (WHC), Daejeon, Korea.","DOI":"10.1109\/WHC.2013.6548399"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5001","DOI":"10.1109\/JSEN.2015.2432127","article-title":"Novel Tactile-SIFT Descriptor for Object Shape Recognition","volume":"15","author":"Luo","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s00221-012-3368-1","article-title":"Exploiting object constancy: Effects of active exploration and shape morphing on similarity judgments of novel objects","volume":"225","author":"Lee","year":"2013","journal-title":"Exp. Brain Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1109\/TOH.2016.2558180","article-title":"Biomimetic Active Touch with Fingertips and Whiskers","volume":"9","author":"Lepora","year":"2016","journal-title":"IEEE Trans. Haptics"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1177\/02783640122068191","article-title":"Feature Detection for Haptic Exploration with Robotic Fingers","volume":"20","author":"Okamura","year":"2001","journal-title":"Int. J. Robot. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lepora, N. (2016). Active Tactile Perception. Scholarpedia of Touch, Atlantis Press.","DOI":"10.2991\/978-94-6239-133-8_11"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TRO.2009.2033627","article-title":"Tactile sensing-from humans to humanoids","volume":"26","author":"Dahiya","year":"2010","journal-title":"IEEE Trans. Robot."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zapata-Impata, B., Gil, P., Torres, F., Zapata-Impata, B.S., Gil, P., and Torres, F. (2019). Learning Spatio Temporal Tactile Features with a ConvLSTM for the Direction Of Slip Detection. Sensors, 19.","DOI":"10.3390\/s19030523"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.robot.2012.07.021","article-title":"Design of a flexible tactile sensor for classification of rigid and deformable objects","volume":"62","author":"Drimus","year":"2014","journal-title":"Robot. Auton. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1016\/j.neuroimage.2017.04.039","article-title":"3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study","volume":"170","author":"Dolz","year":"2018","journal-title":"NeuroImage"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"58213","DOI":"10.1109\/ACCESS.2018.2871977","article-title":"Deep radiomic analysis of MRI related to Alzheimer\u2019s Disease","volume":"6","author":"Chaddad","year":"2018","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gandarias, J.M., Pastor, F., Garc\u00eda-Cerezo, A.J., and G\u00f3mez-de Gabriel, J.M. (2019, January 9\u201312). Active Tactile Recognition of Deformable Objects with 3D Convolutional Neural Networks. Proceedings of the IEEE World Haptics Conference (WHC), Tokyo, Japan.","DOI":"10.1109\/WHC.2019.8816162"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Feng, D., Kaboli, M., and Cheng, G. (2018). Active Prior Tactile Knowledge Transfer for Learning Tactual Properties of New Objects. Sensors, 18.","DOI":"10.3390\/s18020634"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TRO.2018.2830364","article-title":"Robust Tactile Descriptors for Discriminating Objects From Textural Properties via Artificial Robotic Skin","volume":"34","author":"Kaboli","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Baishya, S.S., and Bauml, B. (2016, January 9\u201314). Robust material classification with a tactile skin using deep learning. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7758088"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1109\/TRO.2011.2127110","article-title":"Majority voting: Material classification by tactile sensing using surface texture","volume":"27","author":"Jamali","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, H., Song, X., Nanayakkara, T., Seneviratne, L.D., and Althoefer, K. (2012, January 14\u201318). A computationally fast algorithm for local contact shape and pose classification using a tactile array sensor. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224872"},{"key":"ref_41","unstructured":"Martinez-Hernandez, U., Dodd, T.J., and Prescott, T.J. (2017). Feeling the Shape: Active Exploration Behaviors for Object Recognition With a Robotic Hand. IEEE Trans. Syst. Man Cybern. Syst., 1\u201310."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Yi, Z., Calandra, R., Veiga, F., van Hoof, H., Hermans, T., Zhang, Y., and Peters, J. (2016, January 9\u201314). Active tactile object exploration with gaussian processes. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759723"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Corradi, T., Hall, P., and Iravani, P. (2015, January 26\u201330). Bayesian tactile object recognition: Learning and recognising objects using a new inexpensive tactile sensor. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139744"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Albini, A., Denei, S., and Cannata, G. (2017, January 24\u201328). Human Hand Recognition From Robotic Skin Measurements in Human-Robot Physical Interactions. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206300"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Gandarias, J.M., Garc\u00eda-Cerezo, A.J., and G\u00f3mez-de Gabriel, J.M. (2019). CNN-based Methods for Object Recognition with High-Resolution Tactile Sensors. IEEE Sens. J.","DOI":"10.1109\/JSEN.2019.2912968"},{"key":"ref_46","unstructured":"Falco, P., Lu, S., Cirillo, A., Natale, C., Pirozzi, S., and Lee, D. (June, January 29). Cross-modal visuo-tactile object recognition using robotic active exploration. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Singapore."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Luo, S., Liu, X., Althoefer, K., and Liu, H. (2015, January 24\u201327). Tactile object recognition with semi-supervised learning. Proceedings of the International Conference on Intelligent Robotics and Applications (ICIRA), Portsmouth, UK.","DOI":"10.1007\/978-3-319-22876-1_2"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Khasnobish, A., Jati, A., Singh, G., Bhattacharyya, S., Konar, A., Tibarewala, D., Kim, E., and Nagar, A.K. (2012, January 10\u201315). Object-shape recognition from tactile images using a feed-forward neural network. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Brisbane, QLD, Australia.","DOI":"10.1109\/IJCNN.2012.6252593"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Schmitz, A., Bansho, Y., Noda, K., Iwata, H., Ogata, T., and Sugano, S. (2014, January 18\u201320). Tactile object recognition using deep learning and dropout. Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Madrid, Spain.","DOI":"10.1109\/HUMANOIDS.2014.7041493"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/72.554195","article-title":"Face recognition: A convolutional neural-network approach","volume":"8","author":"Lawrence","year":"1997","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_51","unstructured":"Madry, M., Bo, L., Kragic, D., and Fox, D. (June, January 31). ST-HMP: Unsupervised Spatio-Temporal feature learning for tactile data. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1109\/TIM.2016.2514779","article-title":"Object Recognition Using Tactile Measurements: Kernel Sparse Coding Methods","volume":"65","author":"Liu","year":"2016","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Kerzel, M., Ali, M., Ng, H.G., and Wermter, S. (2017, January 14\u201319). Haptic material classification with a multi-channel neural network. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7965887"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 11\u201318). Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Birglen, L., Lalibert\u00e9, T., and Gosselin, C.M. (2007). Underactuated Robotic Hands, Springer.","DOI":"10.1007\/978-3-540-77459-4"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/www.mdpi.com\/1424-8220\/19\/24\/5356\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:40:21Z","timestamp":1760190021000},"score":1,"resource":{"primary":{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/www.mdpi.com\/1424-8220\/19\/24\/5356"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,5]]},"references-count":55,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["s19245356"],"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.3390\/s19245356","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,5]]}}}