Boot log: qemu_arm64-virt-gicv3-uefi
- Kernel Warnings: 0
- Warnings: 0
- Boot result: FAIL
- Kernel Errors: 0
- Errors: 2
1 10:09:44.603449 lava-dispatcher, installed at version: 2023.06
2 10:09:44.603597 start: 0 validate
3 10:09:44.603693 Start time: 2023-08-14 10:09:44.603686+00:00 (UTC)
4 10:09:44.604232 Validating that http://storage.kernelci.org/images/uefi/edk2-stable202005/arm64/QEMU_EFI.fd exists
5 10:09:44.897967 Validating that http://storage.kernelci.org/cip-gitlab/ci-pavel-linux-test/v4.19.288-cip101-1117-g1f3468542aa9e/arm64/defconfig+arm64-chromebook/gcc-10/kernel/Image exists
6 10:09:45.047582 Validating that http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz exists
7 10:09:45.194434 Calling: 'docker' 'pull' 'kernelci/qemu'
8 10:09:45.211447 >> Using default tag: latest
9 10:09:46.189448 >> latest: Pulling from kernelci/qemu
10 10:09:46.192567 >> Digest: sha256:f32e45c3c1cfd765e1df3fdcb13b1a3a8b04f064750cd2257b85b3d0cb1a7bfc
11 10:09:46.192692 >> Status: Image is up to date for kernelci/qemu:latest
12 10:09:46.195574 >> docker.io/kernelci/qemu:latest
13 10:09:46.196719 Returned 0 in 1 seconds
14 10:09:46.296916 Calling: 'docker' 'pull' 'kernelci/qemu'
15 10:09:46.312588 >> Using default tag: latest
16 10:09:47.300193 >> latest: Pulling from kernelci/qemu
17 10:09:47.304946 >> Digest: sha256:f32e45c3c1cfd765e1df3fdcb13b1a3a8b04f064750cd2257b85b3d0cb1a7bfc
18 10:09:47.305702 >> Status: Image is up to date for kernelci/qemu:latest
19 10:09:47.312145 >> docker.io/kernelci/qemu:latest
20 10:09:47.318434 Returned 0 in 1 seconds
21 10:09:47.445133 Calling: 'docker' 'run' '--rm' '--init' 'kernelci/qemu' 'qemu-system-aarch64' '--version'
22 10:09:48.198943 >> QEMU emulator version 7.2.2 (Debian 1:7.2+dfsg-7~bpo11+1)
23 10:09:48.199290 >> Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
24 10:09:48.602138 Returned 0 in 1 seconds
25 10:09:48.703859 validate duration: 4.10
27 10:09:48.705381 start: 1 deployimages (timeout 00:03:00) [common]
28 10:09:48.705876 start: 1.1 lava-overlay (timeout 00:03:00) [common]
29 10:09:48.706831 [common] Preparing overlay tarball in /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52
30 10:09:48.707446 makedir: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin
31 10:09:48.707874 makedir: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/tests
32 10:09:48.708271 makedir: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/results
33 10:09:48.708729 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-add-keys
34 10:09:48.709375 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-add-sources
35 10:09:48.709884 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-background-process-start
36 10:09:48.710370 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-background-process-stop
37 10:09:48.710818 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-common-functions
38 10:09:48.711258 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-echo-ipv4
39 10:09:48.711699 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-install-packages
40 10:09:48.712142 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-installed-packages
41 10:09:48.712582 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-os-build
42 10:09:48.713139 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-probe-channel
43 10:09:48.713721 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-probe-ip
44 10:09:48.714176 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-target-ip
45 10:09:48.714605 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-target-mac
46 10:09:48.715044 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-target-storage
47 10:09:48.715482 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-case
48 10:09:48.715856 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-event
49 10:09:48.716228 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-feedback
50 10:09:48.716605 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-raise
51 10:09:48.716979 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-reference
52 10:09:48.717335 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-runner
53 10:09:48.717674 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-set
54 10:09:48.717997 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-test-shell
55 10:09:48.718330 Updating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-install-packages (oe)
56 10:09:48.718750 Updating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/bin/lava-installed-packages (oe)
57 10:09:48.719061 Creating /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/environment
58 10:09:48.719313 LAVA metadata
59 10:09:48.719483 - LAVA_JOB_ID=11283140
60 10:09:48.719643 - LAVA_DISPATCHER_IP=192.168.122.1
61 10:09:48.719914 start: 1.1.1 lava-vland-overlay (timeout 00:03:00) [common]
62 10:09:48.720080 skipped lava-vland-overlay
63 10:09:48.720264 end: 1.1.1 lava-vland-overlay (duration 00:00:00) [common]
64 10:09:48.720443 start: 1.1.2 lava-multinode-overlay (timeout 00:03:00) [common]
65 10:09:48.720596 skipped lava-multinode-overlay
66 10:09:48.720766 end: 1.1.2 lava-multinode-overlay (duration 00:00:00) [common]
67 10:09:48.720947 start: 1.1.3 test-definition (timeout 00:03:00) [common]
68 10:09:48.721110 Loading test definitions
69 10:09:48.721308 start: 1.1.3.1 inline-repo-action (timeout 00:03:00) [common]
70 10:09:48.721457 Using /lava-11283140 at stage 0
71 10:09:48.722125 uuid=11283140_1.1.3.1 testdef=None
72 10:09:48.722288 end: 1.1.3.1 inline-repo-action (duration 00:00:00) [common]
73 10:09:48.722421 start: 1.1.3.2 test-overlay (timeout 00:03:00) [common]
74 10:09:48.723307 end: 1.1.3.2 test-overlay (duration 00:00:00) [common]
76 10:09:48.723652 start: 1.1.3.3 test-install-overlay (timeout 00:03:00) [common]
77 10:09:48.724553 end: 1.1.3.3 test-install-overlay (duration 00:00:00) [common]
79 10:09:48.724878 start: 1.1.3.4 test-runscript-overlay (timeout 00:03:00) [common]
80 10:09:48.725580 runner path: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/0/tests/0_dmesg test_uuid 11283140_1.1.3.1
81 10:09:48.725747 end: 1.1.3.4 test-runscript-overlay (duration 00:00:00) [common]
83 10:09:48.725986 start: 1.1.3.5 inline-repo-action (timeout 00:03:00) [common]
84 10:09:48.726053 Using /lava-11283140 at stage 1
85 10:09:48.726336 uuid=11283140_1.1.3.5 testdef=None
86 10:09:48.726425 end: 1.1.3.5 inline-repo-action (duration 00:00:00) [common]
87 10:09:48.726496 start: 1.1.3.6 test-overlay (timeout 00:03:00) [common]
88 10:09:48.726895 end: 1.1.3.6 test-overlay (duration 00:00:00) [common]
90 10:09:48.727077 start: 1.1.3.7 test-install-overlay (timeout 00:03:00) [common]
91 10:09:48.727554 end: 1.1.3.7 test-install-overlay (duration 00:00:00) [common]
93 10:09:48.727723 start: 1.1.3.8 test-runscript-overlay (timeout 00:03:00) [common]
94 10:09:48.728138 runner path: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/1/tests/1_bootrr test_uuid 11283140_1.1.3.5
95 10:09:48.728246 end: 1.1.3.8 test-runscript-overlay (duration 00:00:00) [common]
97 10:09:48.728389 Creating lava-test-runner.conf files
98 10:09:48.728428 Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/0 for stage 0
99 10:09:48.728481 - 0_dmesg
100 10:09:48.728531 Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/11283140/lava-overlay-6_g8dp52/lava-11283140/1 for stage 1
101 10:09:48.728581 - 1_bootrr
102 10:09:48.728636 end: 1.1.3 test-definition (duration 00:00:00) [common]
103 10:09:48.728689 start: 1.1.4 compress-overlay (timeout 00:03:00) [common]
104 10:09:48.734059 end: 1.1.4 compress-overlay (duration 00:00:00) [common]
105 10:09:48.734207 start: 1.1.5 persistent-nfs-overlay (timeout 00:03:00) [common]
106 10:09:48.734268 end: 1.1.5 persistent-nfs-overlay (duration 00:00:00) [common]
107 10:09:48.734319 end: 1.1 lava-overlay (duration 00:00:00) [common]
108 10:09:48.734369 start: 1.2 apply-overlay-guest (timeout 00:03:00) [common]
109 10:09:48.734415 Overlay: /var/lib/lava/dispatcher/tmp/11283140/compress-overlay-k3eyu859/overlay-1.1.4.tar.gz
110 10:09:51.526597 end: 1.2 apply-overlay-guest (duration 00:00:03) [common]
112 10:09:51.528978 start: 1.3 deploy-device-env (timeout 00:02:57) [common]
113 10:09:51.529459 end: 1.3 deploy-device-env (duration 00:00:00) [common]
114 10:09:51.529892 start: 1.4 download-retry (timeout 00:02:57) [common]
115 10:09:51.530306 start: 1.4.1 http-download (timeout 00:02:57) [common]
116 10:09:51.531104 downloading http://storage.kernelci.org/images/uefi/edk2-stable202005/arm64/QEMU_EFI.fd
117 10:09:51.531443 saving as /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/bios/QEMU_EFI.fd
118 10:09:51.531744 total size: 2097152 (2 MB)
119 10:09:51.532060 No compression specified
120 10:09:51.828404 progress 1 % (0 MB)
121 10:09:52.120274 progress 6 % (0 MB)
122 10:09:52.267644 progress 12 % (0 MB)
123 10:09:52.270634 progress 17 % (0 MB)
124 10:09:52.414481 progress 23 % (0 MB)
125 10:09:52.417059 progress 28 % (0 MB)
126 10:09:52.417559 progress 34 % (0 MB)
127 10:09:52.559525 progress 39 % (0 MB)
128 10:09:52.562752 progress 45 % (0 MB)
129 10:09:52.563266 progress 50 % (1 MB)
130 10:09:52.565297 progress 56 % (1 MB)
131 10:09:52.568855 progress 62 % (1 MB)
132 10:09:52.570986 progress 67 % (1 MB)
133 10:09:52.704587 progress 73 % (1 MB)
134 10:09:52.707657 progress 78 % (1 MB)
135 10:09:52.708760 progress 84 % (1 MB)
136 10:09:52.709139 progress 89 % (1 MB)
137 10:09:52.709618 progress 95 % (1 MB)
138 10:09:52.709991 progress 100 % (2 MB)
139 10:09:52.710146 2 MB downloaded in 1.18 s (1.70 MB/s)
140 10:09:52.710256 end: 1.4.1 http-download (duration 00:00:01) [common]
142 10:09:52.710410 end: 1.4 download-retry (duration 00:00:01) [common]
143 10:09:52.710459 start: 1.5 download-retry (timeout 00:02:56) [common]
144 10:09:52.710509 start: 1.5.1 http-download (timeout 00:02:56) [common]
145 10:09:52.710611 downloading http://storage.kernelci.org/cip-gitlab/ci-pavel-linux-test/v4.19.288-cip101-1117-g1f3468542aa9e/arm64/defconfig+arm64-chromebook/gcc-10/kernel/Image
146 10:09:52.710652 saving as /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/kernel/Image
147 10:09:52.710687 total size: 27607552 (26 MB)
148 10:09:52.710722 No compression specified
149 10:09:52.874842 progress 0 % (0 MB)
150 10:09:53.037724 progress 5 % (1 MB)
151 10:09:53.323269 progress 10 % (2 MB)
152 10:09:53.468182 progress 15 % (3 MB)
153 10:09:53.611458 progress 20 % (5 MB)
154 10:09:53.624303 progress 25 % (6 MB)
155 10:09:53.766152 progress 30 % (7 MB)
156 10:09:53.911333 progress 35 % (9 MB)
157 10:09:53.926225 progress 40 % (10 MB)
158 10:09:54.062728 progress 45 % (11 MB)
159 10:09:54.200803 progress 50 % (13 MB)
160 10:09:54.340527 progress 55 % (14 MB)
161 10:09:54.355869 progress 60 % (15 MB)
162 10:09:54.497380 progress 65 % (17 MB)
163 10:09:54.634404 progress 70 % (18 MB)
164 10:09:54.655853 progress 75 % (19 MB)
165 10:09:54.796329 progress 80 % (21 MB)
166 10:09:54.925813 progress 85 % (22 MB)
167 10:09:54.948347 progress 90 % (23 MB)
168 10:09:55.087345 progress 95 % (25 MB)
169 10:09:55.097131 progress 100 % (26 MB)
170 10:09:55.097279 26 MB downloaded in 2.39 s (11.03 MB/s)
171 10:09:55.097392 end: 1.5.1 http-download (duration 00:00:02) [common]
173 10:09:55.097553 end: 1.5 download-retry (duration 00:00:02) [common]
174 10:09:55.097604 start: 1.6 download-retry (timeout 00:02:54) [common]
175 10:09:55.097652 start: 1.6.1 http-download (timeout 00:02:54) [common]
176 10:09:55.097740 Not decompressing ramdisk as can be used compressed.
177 10:09:55.097796 downloading http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz
178 10:09:55.097832 saving as /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/ramdisk/rootfs.cpio.gz
179 10:09:55.097867 total size: 8181372 (7 MB)
180 10:09:55.097903 No compression specified
181 10:09:55.245131 progress 0 % (0 MB)
182 10:09:55.254664 progress 5 % (0 MB)
183 10:09:55.260321 progress 10 % (0 MB)
184 10:09:55.406375 progress 15 % (1 MB)
185 10:09:55.414681 progress 20 % (1 MB)
186 10:09:55.419041 progress 25 % (1 MB)
187 10:09:55.421443 progress 30 % (2 MB)
188 10:09:55.423220 progress 35 % (2 MB)
189 10:09:55.424524 progress 40 % (3 MB)
190 10:09:55.552209 progress 45 % (3 MB)
191 10:09:55.562209 progress 50 % (3 MB)
192 10:09:55.568490 progress 55 % (4 MB)
193 10:09:55.846898 progress 60 % (4 MB)
194 10:09:55.857710 progress 65 % (5 MB)
195 10:09:55.863052 progress 70 % (5 MB)
196 10:09:55.866210 progress 75 % (5 MB)
197 10:09:55.868079 progress 80 % (6 MB)
198 10:09:55.869602 progress 85 % (6 MB)
199 10:09:55.999780 progress 90 % (7 MB)
200 10:09:56.007446 progress 95 % (7 MB)
201 10:09:56.140712 progress 100 % (7 MB)
202 10:09:56.141947 7 MB downloaded in 1.04 s (7.47 MB/s)
203 10:09:56.142637 end: 1.6.1 http-download (duration 00:00:01) [common]
205 10:09:56.143798 end: 1.6 download-retry (duration 00:00:01) [common]
206 10:09:56.144173 end: 1 deployimages (duration 00:00:07) [common]
207 10:09:56.144550 start: 2 boot-image-retry (timeout 00:05:00) [common]
208 10:09:56.144913 start: 2.1 boot-qemu-image (timeout 00:05:00) [common]
209 10:09:56.145302 start: 2.1.1 execute-qemu (timeout 00:05:00) [common]
210 10:09:56.145814 Extending command line for qcow2 test overlay
211 10:09:56.147346 Pulling docker image
212 10:09:56.147711 Calling: 'docker' 'pull' 'kernelci/qemu'
213 10:09:56.169845 >> Using default tag: latest
214 10:09:57.192457 >> latest: Pulling from kernelci/qemu
215 10:09:57.196806 >> Digest: sha256:f32e45c3c1cfd765e1df3fdcb13b1a3a8b04f064750cd2257b85b3d0cb1a7bfc
216 10:09:57.197270 >> Status: Image is up to date for kernelci/qemu:latest
217 10:09:57.201069 >> docker.io/kernelci/qemu:latest
218 10:09:57.206613 Returned 0 in 1 seconds
219 10:09:57.307478 Calling: 'docker' 'pull' 'kernelci/qemu'
220 10:09:57.330270 >> Using default tag: latest
221 10:09:58.354154 >> latest: Pulling from kernelci/qemu
222 10:09:58.357265 >> Digest: sha256:f32e45c3c1cfd765e1df3fdcb13b1a3a8b04f064750cd2257b85b3d0cb1a7bfc
223 10:09:58.357410 >> Status: Image is up to date for kernelci/qemu:latest
224 10:09:58.360387 >> docker.io/kernelci/qemu:latest
225 10:09:58.361668 Returned 0 in 1 seconds
226 10:09:58.480820 Boot command: docker run --network=host --cap-add=NET_ADMIN --interactive --tty --rm --init --name=lava-docker-qemu-11283140-2.1.1-d3no9s4c4j --device=/dev/kvm --device=/dev/net/tun --mount=type=bind,source=/var/lib/lava/dispatcher/tmp,destination=/var/lib/lava/dispatcher/tmp kernelci/qemu qemu-system-aarch64 -cpu max,pauth-impdef=on -machine virt,gic-version=3,mte=on,accel=tcg -nographic -net nic,model=virtio,macaddr=DE:AD:BE:EF:AE:1D -net bridge,br=virbr0 -m 1024 -monitor none -bios /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/bios/QEMU_EFI.fd -kernel /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/kernel/Image -append "console=ttyAMA0,115200 root=/dev/ram0 debug verbose console_msg_format=syslog earlycon deferred_probe_timeout=60" -initrd /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/ramdisk/rootfs.cpio.gz -drive format=qcow2,file=/var/lib/lava/dispatcher/tmp/11283140/apply-overlay-guest-tnrzq5hn/lava-guest.qcow2,media=disk,if=virtio,id=lavatest
227 10:09:58.483780 started a shell command
228 10:09:58.484118 end: 2.1.1 execute-qemu (duration 00:00:02) [common]
229 10:09:58.484212 end: 2.1 boot-qemu-image (duration 00:00:02) [common]
230 10:09:58.484287 start: 2.2 auto-login-action (timeout 00:04:58) [common]
231 10:09:58.484368 Setting prompt string to ['Linux version [0-9]']
232 10:09:58.484425 auto-login-action: Wait for prompt ['Linux version [0-9]'] (timeout 00:05:00)
233 10:09:59.519517 [2J[01;01H[=3h[2J[01;01H[0m[35m[40m[0m[37m[40mEFI stub: Booting Linux Kernel...
234 10:09:59.519683
235 10:09:59.529603 EFI stub: Generating empty DTB
236 10:09:59.529797
237 10:09:59.533221 EFI stub: Exiting boot services and installing virtual address map...
238 10:09:59.533295
240 10:14:56.484621 end: 2.2 auto-login-action (duration 00:04:58) [common]
242 10:14:56.484778 boot-image-retry failed: 1 of 1 attempts. 'auto-login-action timed out after 298 seconds'
243 10:14:56.484827 Stopping the qemu container lava-docker-qemu-11283140-2.1.1-d3no9s4c4j
245 10:14:56.724069 end: 2 boot-image-retry (duration 00:05:01) [common]
247 10:14:56.724359 Cleaning after the job
248 10:14:56.724451 Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/bios
249 10:14:56.724766 Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/kernel
250 10:14:56.727048 Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11283140/deployimages-hs_81dos/ramdisk
251 10:14:56.727848 Stopping the qemu container lava-docker-qemu-11283140-2.1.1-d3no9s4c4j
252 10:14:56.743783 start: 5.1 power-off (timeout 00:00:30) [common]
253 10:14:56.744055 end: 5.1 power-off (duration 00:00:00) [common]
254 10:14:56.744135 start: 5.2 read-feedback (timeout 00:10:00) [common]
255 10:14:56.744349 Listened to connection for namespace 'common' for up to 1s
256 10:14:56.744489 Finalising connection for namespace 'common'
258 10:14:56.844883 poweroff
259 10:14:56.845323 Already disconnected
260 10:14:56.845402 poweroff
261 10:14:56.945746 end: 5.2 read-feedback (duration 00:00:00) [common]
262 10:14:56.945891 Root tmp directory removed at /var/lib/lava/dispatcher/tmp/11283140
263 10:14:56.946578 JobError: Your job cannot terminate cleanly.