secondary-storage: delete temp directory while deleting entity download url #12562

shwstppr · 2026-01-31T08:50:30Z

Description

When expired entity URLs are cleaned up, orphan directories are left behind. This change tries to delete the base temporary directory while cleanup.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Download link for a volume expires and server calls DeleteEntityDownloadURLCommand

Before

2026-01-31T07:36:09,043 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-5:[]) Request:Seq 4-7227995926952935435:  { Cmd , MgmtId: 90520731363323, via: 4, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.storage.DeleteEntityDownloadURLCommand":{"path":"volumes/2/4/97b241b4-4d31-469f-b4a4-a2fcab2606ff.qcow2","extractUrl":"http://172.120.1.2/userdata/88a99588-c0e0-4691-96c7-c7e6534d9fbe/t-dd.qcow2","type":"VOLUME","parentPath":"c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065","accountId":"0","wait":"0","bypassHostMaintenance":"false"}}] }
2026-01-31T07:36:09,044 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-5:[]) Processing command: com.cloud.agent.api.storage.DeleteEntityDownloadURLCommand
2026-01-31T07:36:09,044 DEBUG [storage.resource.NfsSecondaryStorageResource] (AgentRequest-Handler-5:[]) Executing command "DeleteEntityDownloadURLCommand" [{"path":"volumes/2/4/97b241b4-4d31-469f-b4a4-a2fcab2606ff.qcow2","extractUrl":"http://172.120.1.2/userdata/88a99588-c0e0-4691-96c7-c7e6534d9fbe/t-dd.qcow2","type":"VOLUME","parentPath":"c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065","accountId":0,"contextMap":{},"wait":0,"bypassHostMaintenance":false}].
2026-01-31T07:36:09,044 WARN  [storage.template.UploadManagerImpl] (AgentRequest-Handler-5:[]) handleDeleteEntityDownloadURLCommand Path:volumes/2/4/97b241b4-4d31-469f-b4a4-a2fcab2606ff.qcow2 Type:VOLUME
2026-01-31T07:36:09,045 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-5:[]) Executing command [/bin/bash -c unlink /var/www/html/userdata/88a99588-c0e0-4691-96c7-c7e6534d9fbe/t-dd.qcow2 ].
2026-01-31T07:36:09,054 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-5:[]) Successfully executed process [5938] for command [/bin/bash -c unlink /var/www/html/userdata/88a99588-c0e0-4691-96c7-c7e6534d9fbe/t-dd.qcow2 ].
2026-01-31T07:36:09,054 WARN  [storage.template.UploadManagerImpl] (AgentRequest-Handler-5:[]) Deleted volume /mnt/SecStorage/c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065/volumes/2/4/97b241b4-4d31-469f-b4a4-a2fcab2606ff.qcow2
2026-01-31T07:36:09,055 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-5:[]) Executing command [/bin/bash -c rm -rf /mnt/SecStorage/c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065/volumes/2/4/97b241b4-4d31-469f-b4a4-a2fcab2606ff.qcow2 ].
2026-01-31T07:36:09,063 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-5:[]) Successfully executed process [5939] for command [/bin/bash -c rm -rf /mnt/SecStorage/c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065/volumes/2/4/97b241b4-4d31-469f-b4a4-a2fcab2606ff.qcow2 ].
2026-01-31T07:36:09,063 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-5:[]) Seq 4-7227995926952935435:  { Ans: , MgmtId: 90520731363323, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":"true","details":"","wait":"0","bypassHostMaintenance":"false"}}]


root@s-2-VM:/var/www/html/userdata# ls -lrt
total 20
drwxr-xr-x 2 www-data www-data 4096 Jan 31 06:08 9142b6e3-f6c6-48de-86b6-6003eb147d2c
drwxr-xr-x 2 www-data www-data 4096 Jan 31 06:10 85378760-8b68-4c60-81fc-11ddbe59fd85
drwxr-xr-x 2 www-data www-data 4096 Jan 31 06:28 a3276a6b-03aa-44d0-aeef-183da3b214c4
drwxr-xr-x 2 www-data www-data 4096 Jan 31 06:35 0cfcf0a0-58f8-471b-8864-9b8a2b754da3
drwxr-xr-x 2 www-data www-data 4096 Jan 31 07:36 88a99588-c0e0-4691-96c7-c7e6534d9fbe
root@s-2-VM:/var/www/html/userdata# ls -lrt 88a99588-c0e0-4691-96c7-c7e6534d9fbe
total 0

After

2026-01-31T19:38:40,034 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-4:[]) Request:Seq 4-3413728517546836680:  { Cmd , MgmtId: 90520731363323, via: 4, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.storage.DeleteEntityDownloadURLCommand":{"path":"volumes/2/4/b5c47a87-6193-4d78-ab1f-1ade1422c296.qcow2","extractUrl":"http://172.120.1.2/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5/t-dd.qcow2","type":"VOLUME","parentPath":"c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065","accountId":"0","wait":"0","bypassHostMaintenance":"false"}}] }
2026-01-31T19:38:40,035 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-4:[]) Processing command: com.cloud.agent.api.storage.DeleteEntityDownloadURLCommand
2026-01-31T19:38:40,035 DEBUG [storage.resource.NfsSecondaryStorageResource] (AgentRequest-Handler-4:[]) Executing command "DeleteEntityDownloadURLCommand" [{"path":"volumes/2/4/b5c47a87-6193-4d78-ab1f-1ade1422c296.qcow2","extractUrl":"http://172.120.1.2/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5/t-dd.qcow2","type":"VOLUME","parentPath":"c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065","accountId":0,"contextMap":{},"wait":0,"bypassHostMaintenance":false}].
2026-01-31T19:38:40,035 WARN  [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) handleDeleteEntityDownloadURLCommand Path:volumes/2/4/b5c47a87-6193-4d78-ab1f-1ade1422c296.qcow2 Type:VOLUME
2026-01-31T19:38:40,036 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) Executing command [/bin/bash -c unlink /var/www/html/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5/t-dd.qcow2 ].
2026-01-31T19:38:40,039 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) Successfully executed process [8782] for command [/bin/bash -c unlink /var/www/html/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5/t-dd.qcow2 ].
2026-01-31T19:38:40,039 INFO  [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) Deleting symlink root directory: d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5 for http://172.120.1.2/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5/t-dd.qcow2
2026-01-31T19:38:40,039 DEBUG [cloud.utils.FileUtil] (AgentRequest-Handler-4:[]) Deleting path: /var/www/html/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5
2026-01-31T19:38:40,040 WARN  [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) Deleted volume /mnt/SecStorage/c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065/volumes/2/4/b5c47a87-6193-4d78-ab1f-1ade1422c296.qcow2
2026-01-31T19:38:40,040 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) Executing command [/bin/bash -c rm -rf /mnt/SecStorage/c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065/volumes/2/4/b5c47a87-6193-4d78-ab1f-1ade1422c296.qcow2 ].
2026-01-31T19:38:40,048 DEBUG [storage.template.UploadManagerImpl] (AgentRequest-Handler-4:[]) Successfully executed process [8783] for command [/bin/bash -c rm -rf /mnt/SecStorage/c794fe3b-bedb-3e8e-a5c6-c5a2b28a2065/volumes/2/4/b5c47a87-6193-4d78-ab1f-1ade1422c296.qcow2 ].
2026-01-31T19:38:40,049 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-4:[]) Seq 4-3413728517546836680:  { Ans: , MgmtId: 90520731363323, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":"true","details":"","wait":"0","bypassHostMaintenance":"false"}}]


root@s-2-VM:~# ls -lrt /var/www/html/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5
ls: cannot access '/var/www/html/userdata/d3a9b2ea-b212-4480-bca4-0dc1d9d3e6a5': No such file or directory

How did you try to break this feature and the system with this change?

url When expired entity URLs are cleaned up, orphan directories are left behind. This change tries to delete the base temporary directory while cleanup. Signed-off-by: Abhishek Kumar <[email protected]>

codecov · 2026-01-31T08:58:28Z

Codecov Report

❌ Patch coverage is 31.42857% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 16.26%. Comparing base (6bed3d4) to head (eef3e49).

Files with missing lines	Patch %	Lines
...cloudstack/storage/template/UploadManagerImpl.java	47.82%	8 Missing and 4 partials ⚠️
utils/src/main/java/com/cloud/utils/FileUtil.java	0.00%	12 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               4.20   #12562   +/-   ##
=========================================
  Coverage     16.26%   16.26%           
- Complexity    13426    13432    +6     
=========================================
  Files          5660     5660           
  Lines        499949   499982   +33     
  Branches      60704    60712    +8     
=========================================
+ Hits          81296    81332   +36     
+ Misses       409581   409572    -9     
- Partials       9072     9078    +6

Flag	Coverage Δ
uitests	`4.16% <ø> (ø)`
unittests	`17.11% <31.42%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

shwstppr · 2026-01-31T10:21:44Z

@blueorangutan package

blueorangutan · 2026-01-31T10:24:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-01-31T11:16:51Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16663

shwstppr · 2026-01-31T19:05:48Z

@blueorangutan test

blueorangutan · 2026-01-31T19:08:05Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-02-01T09:35:08Z

[SF] Trillian test result (tid-15346)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49207 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12562-t15346-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland

clgtm

secondary-storage: delete temp directory while deleting entity download

eef3e49

url When expired entity URLs are cleaned up, orphan directories are left behind. This change tries to delete the base temporary directory while cleanup. Signed-off-by: Abhishek Kumar <[email protected]>

boring-cyborg bot added component:secondary-storage component:ssvm labels Jan 31, 2026

shwstppr mentioned this pull request Jan 31, 2026

extensions: add sync, download functionalities #11814

Open

14 tasks

shwstppr marked this pull request as ready for review January 31, 2026 19:10

abh1sar added this to the 4.20.3 milestone Feb 1, 2026

DaanHoogland approved these changes Feb 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

secondary-storage: delete temp directory while deleting entity download url #12562

secondary-storage: delete temp directory while deleting entity download url #12562

shwstppr commented Jan 31, 2026 •

edited

Loading

Uh oh!

codecov bot commented Jan 31, 2026 •

edited

Loading

Uh oh!

shwstppr commented Jan 31, 2026

Uh oh!

blueorangutan commented Jan 31, 2026

Uh oh!

blueorangutan commented Jan 31, 2026

Uh oh!

shwstppr commented Jan 31, 2026

Uh oh!

blueorangutan commented Jan 31, 2026

Uh oh!

blueorangutan commented Feb 1, 2026

Uh oh!

DaanHoogland left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

secondary-storage: delete temp directory while deleting entity download url #12562

Are you sure you want to change the base?

secondary-storage: delete temp directory while deleting entity download url #12562

Conversation

shwstppr commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

shwstppr commented Jan 31, 2026

Uh oh!

blueorangutan commented Jan 31, 2026

Uh oh!

blueorangutan commented Jan 31, 2026

Uh oh!

shwstppr commented Jan 31, 2026

Uh oh!

blueorangutan commented Jan 31, 2026

Uh oh!

blueorangutan commented Feb 1, 2026

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shwstppr commented Jan 31, 2026 •

edited

Loading

codecov bot commented Jan 31, 2026 •

edited

Loading