out_s3: Add SSE support + validation tests #11409
Closed
+0
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
GabrielYamin
force-pushed
the
s3_output_sse_support
branch
from
0384477 to
26a7ccd
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds server-side encryption support for S3 uploads with SSE-KMS and DSSE-KMS.
Changes
sseconfiguration option:AES256,aws:kms,aws:kms:dssesse_kms_key_idoption to specify a custom KMS key ARNEnter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
Configuration Example
[OUTPUT] Name s3 Match * bucket my-bucket region us-east-1 sse aws:kms sse_kms_key_id arn:aws:kms:us-east-1:123456789:key/my-key-idTesting
Click to expand test results
No SSE
[OUTPUT] Name s3 Match valid_no_sse bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M s3_key_format /flb-test/valid_no_sse.logSSE-KMS (single layer)
[OUTPUT] Name s3 Match valid_sse_kms bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M sse aws:kms s3_key_format /flb-test/valid_sse_kms.logTrying to upload without SSE headers (bucket requires SSE-KMS):
With SSE header:
SSE-S3 (AES256)
[OUTPUT] Name s3 Match valid_sse_aes256 bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M sse AES256 s3_key_format /flb-test/valid_sse_aes256.logDSSE-KMS (dual layer)
[OUTPUT] Name s3 Match valid_sse_kms_dsse bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M sse aws:kms:dsse s3_key_format /flb-test/valid_sse_kms_dsse.logSSE-KMS with specific key ID
[OUTPUT] Name s3 Match valid_sse_kms_with_key bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M sse aws:kms sse_kms_key_id arn:aws:kms:eu-west-1:<ACCOUNT-NUMBER>:key/<KEY-ID> s3_key_format /flb-test/valid_sse_kms_with_key.logDSSE-KMS with specific key ID
[OUTPUT] Name s3 Match valid_sse_kms_dsse_with_key bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M sse aws:kms:dsse sse_kms_key_id arn:aws:kms:eu-west-1:<ACCOUNT-NUMBER>:key/<KEY-ID> s3_key_format /flb-test/valid_sse_kms_dsse_with_key.logInvalid SSE value rejected
[OUTPUT] Name s3 Match invalid_sse_value bucket <BUCKET-NAME> region eu-west-1 upload_timeout 5s use_put_object true total_file_size 1M sse invalid_encryption s3_key_format /flb-test/invalid_sse_value.logValgrind report
Click to expand valgrind report
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.