SSP462 Audi A8 2010 Night Vision Assist
SSP462 Audi A8 2010 Night Vision Assist
SSP462 Audi A8 2010 Night Vision Assist
Introduction
Recent years have seen many innovations in the field of automotive driver assistance systems. Driver assistance systems aid drivers by providing comfort and safety, but without relieving them of the responsibility for their own actions. They are one of the reasons for the decline in the number of fatalities resulting from road accidents, despite the ever-increasing number of road users. Many of these systems help to prevent accidents by alerting the driver to impending hazards. Night Vision Assist, the latest driver assistance system by Audi, has been designed to specifically meet this challenge. In the dark, Night Vision Assist enables the driver to detect persons ahead of the vehicle much sooner than without it. The advantage of early detection is that the driver has more time to react to the hazardous situation ahead. Animals can also be seen on the display even before they come within the range of the vehicle's headlights. The night vision system is, therefore, a real advancement. It not only provides better illumination of the road, but also allows the situation ahead of the vehicle to be interpreted more quickly and accurately in the dark. Its range greatly exceeds that of the main-beam headlights. The function has been implemented using a thermal imaging camera, which means that heat-emitting objects, such as human beings and animals, stand out clearly against their background.
462_001
The above illustration clearly demonstrates the advantages of the Night Vision Assist system for drivers. The pedestrian crossing the road can only be vaguely discerned through the windscreen. A quick glance at the Night Vision Assist display, however, reveals the pedestrian in full size.
The brightly coloured pedestrian clearly stands out against the background and is bracketed in red to alert the driver to the danger of a collision. By detecting the pedestrian sooner, the driver has more time to react appropriately to the hazardous situation ahead.
Contents
How Night Vision Assist works
Reasons for introducing Night Vision Assist ______________________________________________________________________________ 4 Functional description ______________________________________________________________________________________________________ 5 Range of Night Vision Assist________________________________________________________________________________________________ 6 Information on driver responsibility _______________________________________________________________________________________ 6 Function: marking of detected pedestrians _______________________________________________________________________________ 7 Function: driver hazard warning of Night Vision Assist __________________________________________________________________ 9
System components
Night vision system control unit J853 ___________________________________________________________________________________ Installation location of Night Vision Assist control unit_______________________________________________________________ Component protection system of Night Vision Assist control unit ___________________________________________________ Night vision system camera R212 _______________________________________________________________________________________ Camera shutter heater ____________________________________________________________________________________________________ Installation location of Night Vision Assist camera ____________________________________________________________________ Camera range ______________________________________________________________________________________________________________ Mechanism for protection against misuse of thermal imaging camera______________________________________________ 13 13 13 14 14 15 15 15
System overview
Electrical implementation of the function ______________________________________________________________________________ 16 Communication structure_________________________________________________________________________________________________ 17
The Self-Study Programme teaches the basics of the design and function of new models, automotive components or technologies. It is not a Repair Manual. Figures given are for guidance purposes only and refer to the software version valid at the time of preparation of the SSP. For further information about maintenance and repair work, always refer to the current technical literature.
Note
Reference
462_003
Functional description
The new Audi A8 10 is the first vehicle by Audi to be equipped with Night Vision Assist, a driver assistance system that enables the driver to spot potential hazards more quickly at night. The system produces a thermal image of the area ahead of the vehicle on the multi-functional display. An infrared camera built into the Audi rings at the front end of the vehicle is used to capture these images. Because of the heat they give off, human beings and animals show up much more brightly in the image than their surroundings and, therefore, are easy for the driver to spot on the display. If the system classifies an object as a human being, the object in question is additionally marked in colour. The thermal image shows not only living beings, but also the road ahead of the vehicle and the outline of buildings.
462_004
462_005
The thermal image is further analysed by the night vision system control unit. From the images captured, the system predicts the next direction of movement of detected pedestrians. Actual vehicle speed and yaw rate are factored in to predict the next movements of the vehicle.
If a danger of collision is computed on the basis of these two predictions, an acoustic warning signal will be given so that the driver can react to the situation. However, Night Vision Assist does not itself intervene in road traffic situations.
Dipped headlight, right Main beam head lights Night Vision Assist
462_006
462_007
462_008
Persons can be marked if they are within the range of Night Vision Assist and at a certain distance from the vehicle. The distance must not be greater than about 90 m or less than about 15 m. If persons are further than about 90 m away from the vehicle, they will appear in the thermal image as objects too small to be clearly classified as persons. If persons are closer than 15 m away from the vehicle, they will be too large to be unequivocally classified by the system.
The system is faced the following challenges when it comes to the recognition of persons: Both vehicle and pedestrians are in motion The proportions of human beings can vary considerably from one person to another A two-dimensional image is the basis for classification. If a human being is positioned unfavourably in relation to the camera, classification will be more difficult. It is not enough to evaluate only a single image. A continuous stream of images has to be evaluated in real time. Head and limbs may be covered or concealed, making classification difficult or even impossible. Examples: Wearing of a hood or helmet Carrying of an umbrella Wearing of a thermally insulated jacket that allows little body heat to escape
Note To ensure that the camera image on the multi-functional display is of a consistently high quality, the control unit carries out a temperature calibration every 2 minutes. For this purpose, a shutter is moved in front of the image capture chip for 300 ms. The picture on the multi-functional display is briefly interrupted for 300 ms, creating what on close examination can be perceived by the driver as a quick stop-and-go sequence of images.
Recognition of cyclists and animals a) Cyclists Cyclists are normally detected and marked by Night Vision Assist. Due to the bent posture of cyclists and the cyclical bending of the legs, however, it may be the case that the marking is not displayed continuously. Deactivation of "Marking of detected pedestrians" by the system The ambient temperature rises above 28 C The thermal image loses contrast with rising ambient temperature due to the continuously decreasing difference in temperature between human beings and their surroundings. This makes it increasingly difficult for the system to classify pedestrians. For this reason, "Marking of detected pedestrians" is deactivated at ambient temperatures above 28 C. If the ambient temperature then drops to below 25 C again, "Marking of detected pedestrians" is reactivated.
b) Motorcyclists Night Vision Assist was not developed for the detection of motorcyclists, since these road users themselves are adequately illuminated. Hence, motorcyclists are not marked in the thermal image.
The ambient brightness exceeds a predefined threshold level c) Animals Animals still cannot be detected by the system at present and, therefore, are not marked in colour. The classification of animals is a challenge for the future. Of course, animals can already be discerned in thermal images because of the heat they give off, but are not additionally highlighted. If there is sufficient ambient light, pedestrians can easily be detected with the naked eye, so "Marking of detected pedestrians" is deactivated.
Deactivation of "Marking of detected pedestrians" by the driver The driver can deactivate "Marking of detected pedestrians" on the MMI terminal. A deactivated "Marking of detected pedestrians" function is indicated by the adjacent icon, which appears on the thermal image at the top right. Deactivating the "Marking of detected pedestrians" function always deactivates the driver warning function, too.
462_009
!
8
Note It is possible that the "Marking of detected pedestrians" function will be unavailable on some markets due to country-specific restrictions.
462_010
If different content is being shown on the multi-functional display at the time the warning is generated, the colour of the Night Vision Assist icon on taskbar will change from white to red.
The acoustic warning is also given, unless it has been deactivated by the driver. When a driver warning is given, the multi-functional display does not switch over to the Night Vision Assist picture.
462_011
462_012
The acoustic warning "Warning gong" can be deactivated via the MMI terminal. It is deactivated automatically if "Marking of detected pedestrians" has been deactivated at the MMI terminal. The adjacent icon indicates at the top right of the thermal image that the "Warning gong" function is deactivated.
462_013
Note It is possible that the driver alert function will be unavailable on some markets due to country-specific restrictions.
462_014
462_015
10
Navigation
462_016
A deactivated Night Vision Assist system can be recognised from the fact that no Night Vision Assist icon is shown on the taskbar of the multi-functional display.
462_017
462_019
11
Vehicle wallet
On On
Car systems
462_018
Marking of detected pedestrians If this option is activated, all detected pedestrians are marked yellow in the picture. If a warning is given, the colour of the marking changes from yellow to red.
Contrast The contrast level of the thermal image is alsoadjustable on a scale from -9 to +9. This adjustment option is only available if the thermal image is currently being shown on the dash panel insert display. If different information is currently being displayed, the menu option "Contrast" will be greyed out.
Warning gong If this option is activated, an acoustic signal is given and the pedestrian marked red if there is an imminent danger of a collision with a pedestrian. This adjustment option is only available if the option "Marking of detected pedestrians" is activated.
!
12
Note The settings described here are assigned to the ignition key in use when the ignition is turned off and stored in the Night Vision Assist control unit. They are restored the next time this ignition key is used.
System components
Night vision system control unit J853
The night vision system control unit J853 is the electronic central control unit of Night Vision Assist. The control unit performs the following tasks: To process the raw images produced by the Night Vision Assist camera To detect and then mark persons in the thermal image To continuously evaluate camera images and compute the potential danger of a collision with a detected person To give a warning when a danger of collision is detected To transfer processed thermal images to the dash panel insert To receive and process variables and information required for the operation of Night Vision Assist as a CAN Extended user To supply the camera with battery power To continuously diagnose the system and log any detected faults in the fault memory To aid troubleshooting of the Night Vision Assist system by providing data blocks, adaptions and actuator diagnoses To provide the software required for calibrating the system at service centres and during production To perform dynamic calibration under defined conditions while driving To save the customer's Night Vision Assist settings for the ignition key in use
462_020
462_021
13
Protective cover
Camera lens
Heater element
462_023
14
Spray jet
462_024
Camera range
Night Vision Assist has a range of about 300 m. The night vision system camera R212 has a horizontal opening angle of 24.
462_025
15
System overview
Electrical implementation of the function
The night vision system control unit J853 is a "terminal 15" control unit and has a "terminal 15" line and a "terminal 31" line for independent power supply. The control unit communicates with the night vision system camera R212 via two private bus lines. Among other things, diagnostic information, data and commands are transferred across this line. The raw image produced by the camera is transmitted to the control unit via two image transfer lines. Both bus lines and the image transfer lines are commonly shielded. Two supply lines go from the control unit to the camera. The control unit supplies the camera with battery power.
Terminal 15
Terminal 31
Shield Image transfer line 1 Image transfer line 2 Bus line 1 Bus line 2 Camera power supply Camera ground
462_026
A sheathed and unshielded line with two twisted cores goes from the night vision system control unit J853 to the control unit with display in dash panel insert J285. An analogue video signal is transferred to the multi-functional display via this line.
To interchange data with other control units, two CAN Extended lines lead to the data bus diagnostic interface J533. The diagnostic interface interchanges messages between the CAN Extended bus and other bus systems. It therefore allows data to be swapped between different bus systems.
16
Communication structure
To operate the overall night vision system in the vehicle, the night vision system control unit J853 requires a great many variables and a large volume of information. Many of these variables and information are sourced from other control units, which communicate with each other via various LIN, CAN, FlexRay and MOST bus systems.
Light switch E1
Convenience CAN
Data bus diagnostic interface J533 MOST bus Powertrain CAN Information electronics control unit 1
J794
462_027
Night vision system control unit J853 receives various CAN messages containing variables and content required for its operation transfers the processed thermal image to the dash panel insert transfers information to the dash panel insert for display Data bus diagnostic interface J533 forms the interface between various CAN bus systems and FlexRay is the master control unit for the component protection system Control unit with display in dash panel insert J285 displays the image produced by the Night Vision Assist camera outputs an acoustic signal if a warning is given displays driver information relating to the night vision system displays error messages transfers the filtered ambient temperature to the night vision system control unit
Convenience system control unit J393 sends the "terminal 15" bit electronically for validation purposes Onboard power supply control unit J519 LIN master of rain and light detector sensor G397 and light switch E1 Rain and light detector sensor G397 signals the currently measured brightness level Light switch E1 signals that the night vision system button has been pressed signals the current position of the rotary light switch ABS control unit J104 signals the current vehicle speed signals the current yaw rate
Information electronics control unit J794 The customer can make various adjustments to Night Vision Assist via the MMI terminal
17
462_028
Data blocks
The following information can be read out via data blocks: Control unit power supply Current control unit temperature Saved maximum and minimum temperature values of the control unit with time stamp Stored maximum and minimum temperature values of camera with time stamp Current status of the *shutter: open/closed Camera heater On/Off Camera display information Dipped headlights On/Off Number of faulty camera pixels Night Vision Assist button currently pressed/not pressed Number of recorded frames per second Current brightness value of rain-light sensor Roll angle, yaw angle and dive angle of static calibration Yaw angle and dive angle of dynamic calibration Abort condition of last static calibration Time stamp of last static calibration and VIN Reasons for deactivation of pedestrian detection function (with brightness and ambient temperature values) Number of key currently in use Current vehicle speed System and display status information
Preferences ("Marking of detected pedestrians": On/Off; "Warning gong": On/ Off; image contrast) Preferences stored for key 1 Preferences stored for key 2 Preferences stored for key 3 Preferences stored for key 4
Current yaw rate Dynamic calibration: Current status running / not running Distance travelled with active dynamic calibration Calculated dive angle Calculated yaw angle
Night vision system camera power supply Current power consumption of night vision system camera Current camera temperature
* Shutter ... For temperature calibration purposes, the shutter is moved in front of the image capture chip every 2 minutes.
18
Adaptions
The following functions are available under Adaptions: Reset stored minimum and maximum camera temperature Reset stored minimum and maximum temperature of the night vision system control unit Switch Night Vision Assist On/Off
Actuator diagnostics
The following components can be tested using the actuator diagnostics: Night vision system camera heater Mechanical camera shutter Temporary activation of component protection system
In addition, the actuator diagnostics can be used to start a software routine which determines how many camera pixels are faulty. This check takes about between 2 to 3 minutes. A test picture can be viewed on the multi-functional display by means of another actuator diagnosis.
System calibration
After which work or events does the system have to be recalibrated? Replacement of the Night Vision Assist camera Replacement of the camera mounting Replacement or removal of the front bumper If the fault memory of the Night Vision Assist control unit contains the entry "No or wrong basic setting" Adjustment work on the rear axle Which special tools are required for calibration?
The following special tools are required for calibration of the Audi Night Vision Assist system: a diagnostic tester (e.g. VAS 5051 B) a wheel alignment computer (e.g. VAS 6141) a calibrating device, basic kit VAS 6340/1 or calibrating device VAS 6430 a night vision system calibration plate VAS 6430/6 a linear laser VAS 6350/3
After which work or events does the system not have to be recalibrated? Replacement of the night vision system control unit Flashing the night vision system control unit
462_029
Calibrating device, basic kit VAS 6430/1 (also used for calibration of Audi Lane Assist and of Adaptive Cruise Control ACC)
19
462_030
Calibration process To calibrate the night vision system camera, follow these steps: 1. Place the calibration plate VAS 6430/6 on the adjustment beam of calibrating device VAS 6430/1 2. Position the calibrating device VAS 6430/1 at a distance of 120 cm in front of the camera 3. Start the calibration routine for Night Vision Assist in the wheel alignment computer 4. Height adjustment of calibration plate VAS 6430/6: the height of the calibration plate can be set correctly by turning the crank on the back of the calibration panel 5. To compensate for surface unevenness, place spirit level 1 on the calibration plate and make sure it is level 6. Move the calibrating device into the correct transversal position by shifting it sideways (the wheel alignment computer indicates when a suitable position is reached) 7. Make sure spirit levels 1 and 2 are level using the two adjustment screws 8. Again check the height adjustment using the linear laser, and correct it if necessary 9. Activate the heating function of calibration plate VAS 6430/6 10. Select and start the programme "J853 - Calibration" on the diagnostic tester . This is a two-step program: The first step is to mechanically calibrate the roll angle of the Night Vision Assist camera. The camera must be aligned horizontally by turning the calibration screw using a hexagon socket. The required direction of rotation is predefined by the guided fault finding program. The program indicates when the nominal roll angle is reached. The second step involves calculating the yaw and dive angles and storing them in the night vision system camera. This step is carried out automatically.
Results of calibration Calibrating the Night Vision Assist camera produces the following results: A mechanically correct camera roll angle setting A static camera yaw angle which is corrected electronically by the control unit A static camera dive angle which is also corrected electronically
!
20
Note If it is necessary to calibrate Adaptive Cruise Control (ACC) in addition to Night Vision Assist, the set distance to the calibrating device can also be used for calibrating ACC. However, it is important to observe the sequence of calibration: calibrate Night Vision Assist first, then ACC. To calibrate Audi Lane Assist, however, the calibrating device has to be realigned.
Dynamic calibration
Audi Night Vision Assist also has dynamic calibration capability. The dynamic calibration system starts to run whenever a horizon is detected. The conditions for detecting a horizon are best when driving on country roads or motorways. The dynamic calibration system determines deviations in camera yaw and dive angles from the values obtained by the static calibration. These angular deviations are factored into the calculations for electronic correction of the camera image. If the calculated angular deviations in the dynamic calibration exceed a limit value, "No or incorrect basic setting" will be logged in the fault memory entry of the night vision system control unit. This fault memory entry means that a new static calibration must be carried out. One possible reason for this is that the thermal imaging camera has been knocked out of alignment after minor parking collision. The roll angle of the thermal imaging camera, which can be adjusted mechanically, is not an integral part of the dynamic calibration.
21
Test yourself:
One or more answers may be correct.
1.
a) b) c) d)
Via the Car menu of the MMI The function is activated using the main-beam stalk Via a button in the rotary light switch module Night Vision Assist is active at all times, but the customer must decide whether the image is to be shown on the multi-functional display or not
2.
a) b) c) d)
A thermal imaging camera A radar sensor Multiple ultrasonic sensors A video camera
3.
Which adjustment options does the customer have on the MMI terminal?
a) b) c) d)
System activation and deactivation Contrast level of displayed thermal image Acoustic warning of collision hazard On/Off Whether animals are to be marked in the image or not
4.
a) b) c) d)
The ambient temperature is too high The ambient brightness has exceeded a threshold value The vehicle's road speed had exceeded 120 kph "Marking of detected pedestrians" has been deactivated on the MMI terminal
22
SSP 398 Audi lane assist Functional description Displays and operation of the system Electrical components System overview Communication structure System calibration Diagnostics
SSP 413 The Audi Parking System on the Audi A3 Functional description The parking method of the Audi parking system System components Communication structure Diagnostics
SSP 434 The Audi Headlight Assist System Functional description Mode of operation of the Headlight Assist System Displays and operation of the system Implementation of function in vehicle Diagnostics
23
All rights reserved. Technical specifications are subject to change without notice. Copyright AUDI AG I/VK-35 [email protected] AUDI AG D-85045 Ingolstadt Technical status: 10/09 Printed in Germany A10.5S00.66.20
462