Abstract: Regular inspection of concrete surface defects play an important role in security for extending the service life of infrastructures. An intelligent visual interaction method of detecting crack based on eye tracking was proposed, since that manual inspection on concrete is not efficient and computer vision method is easy to misjudge in complex scenarios. Eye tracking technology was first used to extract human eye information to realize the interactive positioning of suspected crack feature information; the trained deep neural network model was then employed to finely recognize and segment the suspected crack areas, and thus identify and extract cracks in the complex scene pictures. Crack class information from the bridge damage dataset CODEBRIM was used to verify present eye-tracking interactive positioning and crack recognition. It was shown that the present method is 90.6% in positioning accuracy and 83.1% in discriminant accuracy, which are higher than the manual and computer vision method does. Therefore, the proposed method can be used for structural health monitoring.