schema: Add upgrade path from 4.17.1.0 to 4.17.2.0 by yadvr · Pull Request #6981 · apache/cloudstack

yadvr · 2022-12-12T18:19:04Z

This implements a blank/noop upgrade path from 4.17.1.0 to 4.17.2.0 which implements DbUpgradeSystemVmTemplate to kick the systemvm template upgrade.

This implements a blank/noop upgrade path from 4.17.1.0 to 4.17.2.0 which implements DbUpgradeSystemVmTemplate to kick the systemvm template upgrade. Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr · 2022-12-12T18:19:52Z

I tested RC1, the DB upgrade path doesn't kick it :( I'll need to cut RC2 tomorrow and we'll need this (or something similar).
@blueorangutan package

blueorangutan · 2022-12-12T18:21:04Z

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with SystemVM template(s). I'll keep you posted as I make progress.

yadvr · 2022-12-12T18:30:35Z

Manually tested upgrading a 4.17.1.0 kvm env with this patch:

2022-12-12 23:58:24,251 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) DB version = 4.17.1.0 Code Version = 4.17.2.0
2022-12-12 23:58:24,252 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Database upgrade must be performed from 4.17.1.0 to 4.17.2.0
2022-12-12 23:58:24,259 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Running upgrade Upgrade41710to41720 to upgrade from 4.17.1.0-4.17.2.0 to 4.17.2.0
2022-12-12 23:58:24,274 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup upgrade Upgrade41710to41720 to upgrade from 4.17.1.0-4.17.2.0 to 4.17.2.0
2022-12-12 23:58:24,289 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Upgrade completed for version 4.17.2.0
2022-12-12 23:58:24,290 DEBUG [c.c.u.d.Upgrade41710to41720] (main:null) (logid:) Updating System Vm template IDs
2022-12-12 23:58:24,355 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating System Vm template IDs
2022-12-12 23:58:24,357 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating XenServer System Vms
2022-12-12 23:58:24,359 WARN  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.17.2 XenServer SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade
2022-12-12 23:58:24,370 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating Hyperv System Vms
2022-12-12 23:58:24,372 WARN  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.17.2 Hyperv SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade
2022-12-12 23:58:24,374 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating Ovm3 System Vms
2022-12-12 23:58:24,375 WARN  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.17.2 Ovm3 SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade
2022-12-12 23:58:24,379 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating VMware System Vms
2022-12-12 23:58:24,380 WARN  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.17.2 VMware SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade
2022-12-12 23:58:24,387 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating LXC System Vms
2022-12-12 23:58:24,388 WARN  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) 4.17.2 LXC SystemVm template not found. Cannot upgrade system Vms hypervisor is not used, so not failing upgrade
2022-12-12 23:58:24,392 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating KVM System Vms
2022-12-12 23:58:24,394 INFO  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Grabbing lock to register templates.
2022-12-12 23:58:25,943 DEBUG [c.c.u.s.Script] (main:null) (logid:) Executing: /bin/bash -c sudo mount -t nfs 192.168.1.10:/export/cloudstack/secondary /tmp/tmp8410995799525833327
2022-12-12 23:58:25,945 DEBUG [c.c.u.s.Script] (main:null) (logid:) Executing while with timeout : 3600000
2022-12-12 23:58:26,007 DEBUG [c.c.u.s.Script] (main:null) (logid:) Execution is successful.
2022-12-12 23:58:26,056 DEBUG [c.c.u.s.Script] (main:null) (logid:) Looking for setup-sysvm-tmplt in the classpath
2022-12-12 23:58:26,056 DEBUG [c.c.u.s.Script] (main:null) (logid:) System resource: null
2022-12-12 23:58:26,057 DEBUG [c.c.u.s.Script] (main:null) (logid:) Classpath resource: file:/usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt
2022-12-12 23:58:26,057 DEBUG [c.c.u.s.Script] (main:null) (logid:) Absolute path =  /usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt
2022-12-12 23:58:26,057 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Executing: /usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt -u 00dcf0ed-00f4-448f-a5b9-8dec5fd7e122 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.17.2-kvm.qcow2.bz2 -h kvm -d /tmp/tmp8410995799525833327/template/tmpl/1/218
2022-12-12 23:58:26,058 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Executing while with timeout : 1800000
2022-12-12 23:58:50,470 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Execution is successful.
2022-12-12 23:58:50,473 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) + [[ ! -u 00dcf0ed-00f4-448f-a5b9-8dec5fd7e122 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.17.2-kvm.qcow2.bz2 -h kvm -d /tmp/tmp8410995799525833327/template/tmpl/1/218 =~ ^-.+ ]]
+ OPTERR=0
+ DISKSPACE=2120000
+ getopts h:f:d:u::# OPTION
+ case $OPTION in
+ uflag=1
+ uuid=00dcf0ed-00f4-448f-a5b9-8dec5fd7e122
+ getopts h:f:d:u::# OPTION
+ case $OPTION in
+ fflag=1
+ tmpltimg=/usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.17.2-kvm.qcow2.bz2
+ getopts h:f:d:u::# OPTION
+ case $OPTION in
+ hyper=kvm
+ getopts h:f:d:u::# OPTION
+ case $OPTION in
+ destdir=/tmp/tmp8410995799525833327/template/tmpl/1/218
+ getopts h:f:d:u::# OPTION
+ [[ kvm == \k\v\m ]]
+ ext=qcow2
++ which qemu-img
+ qemuimgcmd=/usr/bin/qemu-img
+ localfile=00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
+ sudo mkdir -p /tmp/tmp8410995799525833327/template/tmpl/1/218
+ [[ 0 -ne 0 ]]
+ [[ -f /tmp/tmp8410995799525833327/template/tmpl/1/218/template.properties ]]
++ find /tmp/tmp8410995799525833327/template/tmpl/1/218 -name '*.qcow2'
+ destfiles=
+ [[ '' != '' ]]
+ tmpfolder=/tmp/cloud/templates/
+ mkdir -p /tmp/cloud/templates/
+ tmplfile=/tmp/cloud/templates//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
+ sudo touch /tmp/cloud/templates//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
+ [[ 0 -ne 0 ]]
++ df -P /tmp/tmp8410995799525833327/template/tmpl/1/218
++ awk '{print $4}'
++ tail -1
+ destcap=451352576
+ '[' 451352576 -lt 2120000 ']'
++ df -P /tmp/cloud/templates/
++ awk '{print $4}'
++ tail -1
+ localcap=34434924
+ '[' 34434924 -lt 2120000 ']'
+ [[ 1 == \1 ]]
+ sudo cp /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.17.2-kvm.qcow2.bz2 /tmp/cloud/templates//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
+ [[ 0 -ne 0 ]]
+++ dirname /usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt
++ /usr/share/cloudstack-common/scripts/storage/secondary/createtmplt.sh -s 2 -d 'SystemVM Template ( kvm )' -n 00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2 -t /tmp/tmp8410995799525833327/template/tmpl/1/218/ -f /tmp/cloud/templates//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2 -u -v -c
Uncompressing to /tmp/cloud/templates//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2.tmp (type bz2)...could take a long time
Copying to /tmp/tmp8410995799525833327/template/tmpl/1/218//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2...could take a while
+ installrslt=
+ [[ 0 -ne 0 ]]
+ tmpdestdir=/tmp/cloud/templates/
+ '[' qcow2 == ova ']'
+ rm -rf /tmp/cloud/templates//00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2.tmp
+ tmpltfile=/tmp/tmp8410995799525833327/template/tmpl/1/218/00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
++ ls -l /tmp/tmp8410995799525833327/template/tmpl/1/218/00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
++ awk '-F ' '{print $5}'
+ tmpltsize=461307904
+ [[ qcow2 == \q\c\o\w\2 ]]
++ /usr/bin/qemu-img info /tmp/tmp8410995799525833327/template/tmpl/1/218/00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
++ grep -i 'virtual size'
++ sed -ne 's/.*(\([0-9]*\).*/\1/p'
++ xargs
+ vrtmpltsize=5242880000
+ templateId=218
+ sudo touch /tmp/tmp8410995799525833327/template/tmpl/1/218/template.properties
+ echo qcow2=true
+ echo id=218
+ echo public=true
+ echo qcow2.filename=00dcf0ed-00f4-448f-a5b9-8dec5fd7e122.qcow2
+ echo uniquename=routing-218
+ echo qcow2.virtualsize=5242880000
+ echo virtualsize=5242880000
+ echo qcow2.size=461307904
+ sudo cp /tmp/cloud/templates//template.properties /tmp/tmp8410995799525833327/template/tmpl/1/218/template.properties
+ '[' -f /tmp/cloud/templates//template.properties ']'
+ rm -rf /tmp/cloud/templates//template.properties
+ echo 'Successfully installed system VM template /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.17.2-kvm.qcow2.bz2 and template.properties to /tmp/tmp8410995799525833327/template/tmpl/1/218'
Successfully installed system VM template /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.17.2-kvm.qcow2.bz2 and template.properties to /tmp/tmp8410995799525833327/template/tmpl/1/218
+ exit 0

2022-12-12 23:58:50,490 INFO  [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Unmounting store
2022-12-12 23:58:50,491 DEBUG [c.c.u.s.Script] (main:null) (logid:) Executing: /bin/bash -c sudo umount /tmp/tmp8410995799525833327
2022-12-12 23:58:50,493 DEBUG [c.c.u.s.Script] (main:null) (logid:) Executing while with timeout : 3600000
2022-12-12 23:58:50,569 DEBUG [c.c.u.s.Script] (main:null) (logid:) Execution is successful.
2022-12-12 23:58:50,571 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:null) (logid:) Updating System Vm Template IDs Complete
2

blueorangutan · 2022-12-12T18:49:38Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 4866

yadvr · 2022-12-12T19:01:31Z

@blueorangutan test

yadvr · 2022-12-12T19:02:16Z

@blueorangutan test

blueorangutan · 2022-12-12T19:03:03Z

@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

sonarqubecloud · 2022-12-12T19:23:57Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
3 Security Hotspots
4 Code Smells

10.8% Coverage
37.3% Duplication

codecov · 2022-12-12T20:19:00Z

Codecov Report

Merging #6981 (e1eb6c4) into 4.17 (5516f74) will increase coverage by 0.00%.
The diff coverage is 21.05%.

@@            Coverage Diff            @@
##               4.17    #6981   +/-   ##
=========================================
  Coverage     10.36%   10.36%           
- Complexity     6629     6633    +4     
=========================================
  Files          2452     2453    +1     
  Lines        242367   242385   +18     
  Branches      37927    37927           
=========================================
+ Hits          25123    25130    +7     
- Misses       214141   214151   +10     
- Partials       3103     3104    +1

Impacted Files	Coverage Δ
...ava/com/cloud/upgrade/dao/Upgrade41710to41720.java	`11.76% <11.76%> (ø)`
...java/com/cloud/upgrade/DatabaseUpgradeChecker.java	`46.66% <100.00%> (+0.25%)`	⬆️
...ava/com/cloud/upgrade/dao/Upgrade41700to41710.java	`4.76% <100.00%> (ø)`
...dstack/network/contrail/model/ModelObjectBase.java	`28.84% <0.00%> (+7.69%)`	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

harikrishna-patnala

Tested on an environment and it upgraded.

| 38 | 4.17.0.0 | 2022-12-05 06:01:48 | Complete |
| 39 | 4.17.1.0 | 2022-12-05 06:01:48 | Complete |
| 41 | 4.17.2.0 | 2022-12-13 04:21:35 | Complete |

LGTM

weizhouapache

code and testing results look good

blueorangutan · 2022-12-13T07:02:57Z

Trillian test result (tid-5482)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40316 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6981-t5482-kvm-centos7.zip
Smoke tests completed. 101 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

schema: Add upgrade path from 4.17.1.0 to 4.17.2.0

e1eb6c4

This implements a blank/noop upgrade path from 4.17.1.0 to 4.17.2.0 which implements DbUpgradeSystemVmTemplate to kick the systemvm template upgrade. Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr added this to the 4.17.2.0 milestone Dec 12, 2022

yadvr requested review from DaanHoogland, nvazquez and shwstppr December 12, 2022 18:19

yadvr mentioned this pull request Dec 12, 2022

SystemVM Template Upgrade doesn't kick in for NoopDbUpgrade #6980

Closed

yadvr added Severity:BLOCKER Severity:Critical Critical bug component:systemvm-template component:upgrade and removed Severity:Critical Critical bug labels Dec 12, 2022

yadvr requested a review from weizhouapache December 12, 2022 19:01

harikrishna-patnala approved these changes Dec 13, 2022

View reviewed changes

shwstppr approved these changes Dec 13, 2022

View reviewed changes

weizhouapache approved these changes Dec 13, 2022

View reviewed changes

yadvr merged commit d331b2f into apache:4.17 Dec 13, 2022

yadvr deleted the 417-systemvmupgrade branch December 13, 2022 07:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

schema: Add upgrade path from 4.17.1.0 to 4.17.2.0#6981

schema: Add upgrade path from 4.17.1.0 to 4.17.2.0#6981
yadvr merged 1 commit into
apache:4.17from
shapeblue:417-systemvmupgrade

yadvr commented Dec 12, 2022 •

edited

Loading

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

blueorangutan commented Dec 12, 2022

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

blueorangutan commented Dec 12, 2022

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

blueorangutan commented Dec 12, 2022

Uh oh!

sonarqubecloud Bot commented Dec 12, 2022

Uh oh!

codecov Bot commented Dec 12, 2022 •

edited

Loading

Uh oh!

harikrishna-patnala left a comment

Uh oh!

weizhouapache left a comment

Uh oh!

blueorangutan commented Dec 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

yadvr commented Dec 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

blueorangutan commented Dec 12, 2022

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

blueorangutan commented Dec 12, 2022

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

yadvr commented Dec 12, 2022

Uh oh!

blueorangutan commented Dec 12, 2022

Uh oh!

sonarqubecloud Bot commented Dec 12, 2022

Uh oh!

codecov Bot commented Dec 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

harikrishna-patnala left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Dec 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

yadvr commented Dec 12, 2022 •

edited

Loading

codecov Bot commented Dec 12, 2022 •

edited

Loading