Boot log: qemu_arm64-virt-gicv3-uefi

    1 22:37:41.410388  lava-dispatcher, installed at version: 2024.01
    2 22:37:41.410538  start: 0 validate
    3 22:37:41.410634  Start time: 2024-02-06 22:37:41.410629+00:00 (UTC)
    4 22:37:41.411196  Validating that http://storage.kernelci.org/images/uefi/edk2-stable202005/arm64/QEMU_EFI.fd exists
    5 22:37:41.705810  Validating that http://storage.kernelci.org/cip/linux-4.19.y-cip/v4.19.304-cip106-2-g5723c66be3b0/arm64/defconfig+arm64-chromebook/gcc-10/kernel/Image exists
    6 22:37:41.854955  Validating that http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz exists
    7 22:37:42.004420  Calling: 'docker' 'pull' 'kernelci/qemu'
    8 22:37:42.027839  >> Using default tag: latest

    9 22:37:43.115143  >> latest: Pulling from kernelci/qemu

   10 22:37:43.119632  >> Digest: sha256:8ef0108db22dfd5c2d6adf59333c88b96c3887078fbf4af157f9621151f2d1c5

   11 22:37:43.120105  >> Status: Image is up to date for kernelci/qemu:latest

   12 22:37:43.123114  >> docker.io/kernelci/qemu:latest

   13 22:37:43.124621  Returned 0 in 1 seconds
   14 22:37:43.224964  Calling: 'docker' 'pull' 'kernelci/qemu'
   15 22:37:43.258004  >> Using default tag: latest

   16 22:37:44.173599  >> latest: Pulling from kernelci/qemu

   17 22:37:44.177308  >> Digest: sha256:8ef0108db22dfd5c2d6adf59333c88b96c3887078fbf4af157f9621151f2d1c5

   18 22:37:44.177977  >> Status: Image is up to date for kernelci/qemu:latest

   19 22:37:44.182210  >> docker.io/kernelci/qemu:latest

   20 22:37:44.185680  Returned 0 in 0 seconds
   21 22:37:45.499619  qemu, installed at version: 7.2.5, host architecture: x86_64
   22 22:37:45.500168  validate duration: 4.09
   24 22:37:45.500416  start: 1 deployimages (timeout 00:03:00) [common]
   25 22:37:45.500491  start: 1.1 lava-overlay (timeout 00:03:00) [common]
   26 22:37:45.500695  [common] Preparing overlay tarball in /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_
   27 22:37:45.500817  makedir: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin
   28 22:37:45.500885  makedir: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/tests
   29 22:37:45.500944  makedir: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/results
   30 22:37:45.501021  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-add-keys
   31 22:37:45.501127  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-add-sources
   32 22:37:45.501209  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-background-process-start
   33 22:37:45.501285  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-background-process-stop
   34 22:37:45.501355  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-common-functions
   35 22:37:45.501424  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-echo-ipv4
   36 22:37:45.501494  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-install-packages
   37 22:37:45.501561  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-installed-packages
   38 22:37:45.501627  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-os-build
   39 22:37:45.501711  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-probe-channel
   40 22:37:45.501782  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-probe-ip
   41 22:37:45.501849  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-target-ip
   42 22:37:45.501916  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-target-mac
   43 22:37:45.501982  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-target-storage
   44 22:37:45.502053  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-case
   45 22:37:45.502122  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-event
   46 22:37:45.502192  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-feedback
   47 22:37:45.502258  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-raise
   48 22:37:45.502329  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-reference
   49 22:37:45.502400  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-runner
   50 22:37:45.502466  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-set
   51 22:37:45.502538  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-test-shell
   52 22:37:45.502617  Updating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-install-packages (oe)
   53 22:37:45.502740  Updating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/bin/lava-installed-packages (oe)
   54 22:37:45.502832  Creating /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/environment
   55 22:37:45.502914  LAVA metadata
   56 22:37:45.502965  - LAVA_JOB_ID=12709816
   57 22:37:45.503009  - LAVA_DISPATCHER_IP=192.168.122.1
   58 22:37:45.503094  start: 1.1.1 lava-vland-overlay (timeout 00:03:00) [common]
   59 22:37:45.503139  skipped lava-vland-overlay
   60 22:37:45.503188  end: 1.1.1 lava-vland-overlay (duration 00:00:00) [common]
   61 22:37:45.503243  start: 1.1.2 lava-multinode-overlay (timeout 00:03:00) [common]
   62 22:37:45.503284  skipped lava-multinode-overlay
   63 22:37:45.503333  end: 1.1.2 lava-multinode-overlay (duration 00:00:00) [common]
   64 22:37:45.503383  start: 1.1.3 test-definition (timeout 00:03:00) [common]
   65 22:37:45.503435  Loading test definitions
   66 22:37:45.503498  start: 1.1.3.1 inline-repo-action (timeout 00:03:00) [common]
   67 22:37:45.503554  Using /lava-12709816 at stage 0
   68 22:37:45.503811  uuid=12709816_1.1.3.1 testdef=None
   69 22:37:45.503882  end: 1.1.3.1 inline-repo-action (duration 00:00:00) [common]
   70 22:37:45.503941  start: 1.1.3.2 test-overlay (timeout 00:03:00) [common]
   71 22:37:45.504425  end: 1.1.3.2 test-overlay (duration 00:00:00) [common]
   73 22:37:45.504600  start: 1.1.3.3 test-install-overlay (timeout 00:03:00) [common]
   74 22:37:45.505093  end: 1.1.3.3 test-install-overlay (duration 00:00:00) [common]
   76 22:37:45.505274  start: 1.1.3.4 test-runscript-overlay (timeout 00:03:00) [common]
   77 22:37:45.505730  runner path: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/0/tests/0_dmesg test_uuid 12709816_1.1.3.1
   78 22:37:45.505850  end: 1.1.3.4 test-runscript-overlay (duration 00:00:00) [common]
   80 22:37:45.506012  start: 1.1.3.5 inline-repo-action (timeout 00:03:00) [common]
   81 22:37:45.506058  Using /lava-12709816 at stage 1
   82 22:37:45.506278  uuid=12709816_1.1.3.5 testdef=None
   83 22:37:45.506338  end: 1.1.3.5 inline-repo-action (duration 00:00:00) [common]
   84 22:37:45.506393  start: 1.1.3.6 test-overlay (timeout 00:03:00) [common]
   85 22:37:45.506721  end: 1.1.3.6 test-overlay (duration 00:00:00) [common]
   87 22:37:45.506873  start: 1.1.3.7 test-install-overlay (timeout 00:03:00) [common]
   88 22:37:45.507317  end: 1.1.3.7 test-install-overlay (duration 00:00:00) [common]
   90 22:37:45.507485  start: 1.1.3.8 test-runscript-overlay (timeout 00:03:00) [common]
   91 22:37:45.507931  runner path: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/1/tests/1_bootrr test_uuid 12709816_1.1.3.5
   92 22:37:45.508040  end: 1.1.3.8 test-runscript-overlay (duration 00:00:00) [common]
   94 22:37:45.508183  Creating lava-test-runner.conf files
   95 22:37:45.508223  Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/0 for stage 0
   96 22:37:45.508277  - 0_dmesg
   97 22:37:45.508326  Using lava-test-runner path: /var/lib/lava/dispatcher/tmp/12709816/lava-overlay-bnkjpeg_/lava-12709816/1 for stage 1
   98 22:37:45.508376  - 1_bootrr
   99 22:37:45.508429  end: 1.1.3 test-definition (duration 00:00:00) [common]
  100 22:37:45.508483  start: 1.1.4 compress-overlay (timeout 00:03:00) [common]
  101 22:37:45.514449  end: 1.1.4 compress-overlay (duration 00:00:00) [common]
  102 22:37:45.514547  start: 1.1.5 persistent-nfs-overlay (timeout 00:03:00) [common]
  103 22:37:45.514608  end: 1.1.5 persistent-nfs-overlay (duration 00:00:00) [common]
  104 22:37:45.514662  end: 1.1 lava-overlay (duration 00:00:00) [common]
  105 22:37:45.514715  start: 1.2 apply-overlay-guest (timeout 00:03:00) [common]
  106 22:37:45.514763  Overlay: /var/lib/lava/dispatcher/tmp/12709816/compress-overlay-dh59wszx/overlay-1.1.4.tar.gz
  107 22:37:48.183796  end: 1.2 apply-overlay-guest (duration 00:00:03) [common]
  109 22:37:48.184781  start: 1.3 deploy-device-env (timeout 00:02:57) [common]
  110 22:37:48.184939  end: 1.3 deploy-device-env (duration 00:00:00) [common]
  111 22:37:48.185083  start: 1.4 download-retry (timeout 00:02:57) [common]
  112 22:37:48.185230  start: 1.4.1 http-download (timeout 00:02:57) [common]
  113 22:37:48.185545  downloading http://storage.kernelci.org/images/uefi/edk2-stable202005/arm64/QEMU_EFI.fd
  114 22:37:48.185676  saving as /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/bios/QEMU_EFI.fd
  115 22:37:48.185771  total size: 2097152 (2 MB)
  116 22:37:48.185864  No compression specified
  117 22:37:48.478897  progress   1 % (0 MB)
  118 22:37:48.770538  progress   6 % (0 MB)
  119 22:37:48.916632  progress  12 % (0 MB)
  120 22:37:48.919721  progress  17 % (0 MB)
  121 22:37:49.062432  progress  23 % (0 MB)
  122 22:37:49.065431  progress  28 % (0 MB)
  123 22:37:49.068903  progress  34 % (0 MB)
  124 22:37:49.071567  progress  39 % (0 MB)
  125 22:37:49.207869  progress  45 % (0 MB)
  126 22:37:49.210922  progress  50 % (1 MB)
  127 22:37:49.214391  progress  56 % (1 MB)
  128 22:37:49.217779  progress  62 % (1 MB)
  129 22:37:49.220335  progress  67 % (1 MB)
  130 22:37:49.222818  progress  73 % (1 MB)
  131 22:37:49.224367  progress  78 % (1 MB)
  132 22:37:49.353274  progress  84 % (1 MB)
  133 22:37:49.356310  progress  89 % (1 MB)
  134 22:37:49.359775  progress  95 % (1 MB)
  135 22:37:49.496967  progress 100 % (2 MB)
  136 22:37:49.498518  2 MB downloaded in 1.31 s (1.52 MB/s)
  137 22:37:49.499276  end: 1.4.1 http-download (duration 00:00:01) [common]
  139 22:37:49.500410  end: 1.4 download-retry (duration 00:00:01) [common]
  140 22:37:49.500793  start: 1.5 download-retry (timeout 00:02:56) [common]
  141 22:37:49.501123  start: 1.5.1 http-download (timeout 00:02:56) [common]
  142 22:37:49.501643  downloading http://storage.kernelci.org/cip/linux-4.19.y-cip/v4.19.304-cip106-2-g5723c66be3b0/arm64/defconfig+arm64-chromebook/gcc-10/kernel/Image
  143 22:37:49.501701  saving as /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/kernel/Image
  144 22:37:49.501737  total size: 29577728 (28 MB)
  145 22:37:49.501773  No compression specified
  146 22:37:49.649352  progress   0 % (0 MB)
  147 22:37:49.821160  progress   5 % (1 MB)
  148 22:37:49.958092  progress  10 % (2 MB)
  149 22:37:50.101932  progress  15 % (4 MB)
  150 22:37:50.395769  progress  20 % (5 MB)
  151 22:37:50.402878  progress  25 % (7 MB)
  152 22:37:50.543406  progress  30 % (8 MB)
  153 22:37:50.692504  progress  35 % (9 MB)
  154 22:37:50.837397  progress  40 % (11 MB)
  155 22:37:50.981819  progress  45 % (12 MB)
  156 22:37:51.127107  progress  50 % (14 MB)
  157 22:37:51.269258  progress  55 % (15 MB)
  158 22:37:51.418420  progress  60 % (16 MB)
  159 22:37:51.557577  progress  65 % (18 MB)
  160 22:37:51.706167  progress  70 % (19 MB)
  161 22:37:51.851540  progress  75 % (21 MB)
  162 22:37:51.996620  progress  80 % (22 MB)
  163 22:37:52.141007  progress  85 % (24 MB)
  164 22:37:52.282972  progress  90 % (25 MB)
  165 22:37:52.423922  progress  95 % (26 MB)
  166 22:37:52.439163  progress 100 % (28 MB)
  167 22:37:52.439482  28 MB downloaded in 2.94 s (9.60 MB/s)
  168 22:37:52.439679  end: 1.5.1 http-download (duration 00:00:03) [common]
  170 22:37:52.439984  end: 1.5 download-retry (duration 00:00:03) [common]
  171 22:37:52.440084  start: 1.6 download-retry (timeout 00:02:53) [common]
  172 22:37:52.440177  start: 1.6.1 http-download (timeout 00:02:53) [common]
  173 22:37:52.440320  Not decompressing ramdisk as can be used compressed.
  174 22:37:52.440413  downloading http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz
  175 22:37:52.440480  saving as /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/ramdisk/rootfs.cpio.gz
  176 22:37:52.440541  total size: 8181372 (7 MB)
  177 22:37:52.440602  No compression specified
  178 22:37:52.588137  progress   0 % (0 MB)
  179 22:37:52.593331  progress   5 % (0 MB)
  180 22:37:52.594778  progress  10 % (0 MB)
  181 22:37:52.597586  progress  15 % (1 MB)
  182 22:37:52.881795  progress  20 % (1 MB)
  183 22:37:52.885404  progress  25 % (1 MB)
  184 22:37:52.886724  progress  30 % (2 MB)
  185 22:37:52.888125  progress  35 % (2 MB)
  186 22:37:52.889419  progress  40 % (3 MB)
  187 22:37:52.890822  progress  45 % (3 MB)
  188 22:37:52.892116  progress  50 % (3 MB)
  189 22:37:53.030660  progress  55 % (4 MB)
  190 22:37:53.036467  progress  60 % (4 MB)
  191 22:37:53.042652  progress  65 % (5 MB)
  192 22:37:53.202116  progress  70 % (5 MB)
  193 22:37:53.214530  progress  75 % (5 MB)
  194 22:37:53.488341  progress  80 % (6 MB)
  195 22:37:53.499012  progress  85 % (6 MB)
  196 22:37:53.504583  progress  90 % (7 MB)
  197 22:37:53.507864  progress  95 % (7 MB)
  198 22:37:53.509791  progress 100 % (7 MB)
  199 22:37:53.509957  7 MB downloaded in 1.07 s (7.30 MB/s)
  200 22:37:53.510069  end: 1.6.1 http-download (duration 00:00:01) [common]
  202 22:37:53.510245  end: 1.6 download-retry (duration 00:00:01) [common]
  203 22:37:53.510301  end: 1 deployimages (duration 00:00:08) [common]
  204 22:37:53.510361  start: 2 boot-image-retry (timeout 00:05:00) [common]
  205 22:37:53.510416  start: 2.1 boot-qemu-image (timeout 00:05:00) [common]
  206 22:37:53.510465  start: 2.1.1 execute-qemu (timeout 00:05:00) [common]
  207 22:37:53.510546  Extending command line for qcow2 test overlay
  208 22:37:53.510800  Pulling docker image
  209 22:37:53.510857  Calling: 'docker' 'pull' 'kernelci/qemu'
  210 22:37:53.526887  >> Using default tag: latest

  211 22:37:54.496986  >> latest: Pulling from kernelci/qemu

  212 22:37:54.501561  >> Digest: sha256:8ef0108db22dfd5c2d6adf59333c88b96c3887078fbf4af157f9621151f2d1c5

  213 22:37:54.502093  >> Status: Image is up to date for kernelci/qemu:latest

  214 22:37:54.506105  >> docker.io/kernelci/qemu:latest

  215 22:37:54.512395  Returned 0 in 1 seconds
  216 22:37:54.613274  Calling: 'docker' 'pull' 'kernelci/qemu'
  217 22:37:54.635798  >> Using default tag: latest

  218 22:37:55.557098  >> latest: Pulling from kernelci/qemu

  219 22:37:55.562230  >> Digest: sha256:8ef0108db22dfd5c2d6adf59333c88b96c3887078fbf4af157f9621151f2d1c5

  220 22:37:55.562880  >> Status: Image is up to date for kernelci/qemu:latest

  221 22:37:55.566705  >> docker.io/kernelci/qemu:latest

  222 22:37:55.571546  Returned 0 in 0 seconds
  223 22:37:55.691138  Boot command: docker run --network=host --cap-add=NET_ADMIN --interactive --tty --rm --init --name=lava-docker-qemu-12709816-2.1.1-82m42d2b2b --device=/dev/kvm --device=/dev/net/tun --mount=type=bind,source=/var/lib/lava/dispatcher/tmp,destination=/var/lib/lava/dispatcher/tmp kernelci/qemu qemu-system-aarch64 -cpu max,pauth-impdef=on -machine virt,gic-version=3,mte=on,accel=tcg -nographic -net nic,model=virtio,macaddr=DE:AD:BE:EF:AE:1D -net bridge,br=virbr0 -m 1024 -monitor none -bios /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/bios/QEMU_EFI.fd -kernel /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/kernel/Image -append "console=ttyAMA0,115200 root=/dev/ram0 debug verbose console_msg_format=syslog earlycon deferred_probe_timeout=60" -initrd /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/ramdisk/rootfs.cpio.gz -drive format=qcow2,file=/var/lib/lava/dispatcher/tmp/12709816/apply-overlay-guest-2k85p2gt/lava-guest.qcow2,media=disk,if=virtio,id=lavatest
  224 22:37:55.694265  started a shell command
  225 22:37:55.694608  end: 2.1.1 execute-qemu (duration 00:00:02) [common]
  227 22:37:55.694813  end: 2.1 boot-qemu-image (duration 00:00:02) [common]
  228 22:37:55.694886  start: 2.2 auto-login-action (timeout 00:04:58) [common]
  229 22:37:55.694964  Setting prompt string to ['Linux version [0-9]']
  230 22:37:55.695014  auto-login-action: Wait for prompt ['Linux version [0-9]'] (timeout 00:05:00)
  231 22:37:56.657803  [=3hEFI stub: Booting Linux Kernel...

  232 22:37:56.658029  

  233 22:37:56.667367  EFI stub: Generating empty DTB

  234 22:37:56.667573  

  235 22:37:56.670248  EFI stub: Exiting boot services and installing virtual address map...

  236 22:37:56.670388  

  238 22:42:53.695201  end: 2.2 auto-login-action (duration 00:04:58) [common]
  240 22:42:53.695349  boot-image-retry failed: 1 of 1 attempts. 'auto-login-action timed out after 298 seconds'
  241 22:42:53.695397  Stopping the qemu container lava-docker-qemu-12709816-2.1.1-82m42d2b2b
  243 22:42:53.899154  end: 2 boot-image-retry (duration 00:05:00) [common]
  245 22:42:53.899449  Cleaning after the job
  246 22:42:53.899553  Cleaning up download directory: /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/bios
  247 22:42:53.899904  Cleaning up download directory: /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/kernel
  248 22:42:53.902727  Cleaning up download directory: /var/lib/lava/dispatcher/tmp/12709816/deployimages-0hl4n9uf/ramdisk
  249 22:42:53.903471  Stopping the qemu container lava-docker-qemu-12709816-2.1.1-82m42d2b2b
  250 22:42:53.920635  start: 5.1 power-off (timeout 00:00:30) [common]
  251 22:42:53.920938  end: 5.1 power-off (duration 00:00:00) [common]
  252 22:42:53.921020  start: 5.2 read-feedback (timeout 00:10:00) [common]
  253 22:42:53.921273  Listened to connection for namespace 'common' for up to 1s
  254 22:42:53.921433  Finalising connection for namespace 'common'
  256 22:42:54.021706  poweroff

  258 22:42:54.022130  Already disconnected
  259 22:42:54.122410  end: 5.2 read-feedback (duration 00:00:00) [common]
  260 22:42:54.122586  Root tmp directory removed at /var/lib/lava/dispatcher/tmp/12709816
  261 22:42:54.123376  JobError: Your job cannot terminate cleanly.