Reduce memory usage for chunked artifact uploads to MinIO (#31325) (#31338)

Backport #31325 by @bohde

When using the MinIO storage driver for Actions Artifacts, we found that
the chunked artifact required significantly more memory usage to both
upload and merge than the local storage driver. This seems to be related
to hardcoding a value of `-1` for the size to the MinIO client [which
has a warning about memory usage in the respective
docs](https://pkg.go.dev/github.com/minio/minio-go/v7#Client.PutObject).
Specifying the size in both the upload and merge case reduces memory
usage of the MinIO client.

Co-authored-by: Rowan Bohde <rowan.bohde@gmail.com>
Co-authored-by: Kyle D <kdumontnu@gmail.com>

diff --git a/routers/api/actions/artifacts_chunks.go b/routers/api/actions/artifacts_chunks.go
index 3a81724b3adf3d6f346df5b78f11296947cfa790..b0c96585cb9c9751114f3f6ce7936137be4a866a 100644 (file)
--- a/routers/api/actions/artifacts_chunks.go
+++ b/routers/api/actions/artifacts_chunks.go
@@ -39,7 +39,7 @@ func saveUploadChunkBase(st storage.ObjectStorage, ctx *ArtifactContext,
                r = io.TeeReader(r, hasher)
        }
        // save chunk to storage
-       writtenSize, err := st.Save(storagePath, r, -1)
+       writtenSize, err := st.Save(storagePath, r, contentSize)
        if err != nil {
                return -1, fmt.Errorf("save chunk to storage error: %v", err)
        }
@@ -208,7 +208,7 @@ func mergeChunksForArtifact(ctx *ArtifactContext, chunks []*chunkFileItem, st st
 
        // save merged file
        storagePath := fmt.Sprintf("%d/%d/%d.%s", artifact.RunID%255, artifact.ID%255, time.Now().UnixNano(), extension)
-       written, err := st.Save(storagePath, mergedReader, -1)
+       written, err := st.Save(storagePath, mergedReader, artifact.FileCompressedSize)
        if err != nil {
                return fmt.Errorf("save merged file error: %v", err)
        }