-
Notifications
You must be signed in to change notification settings - Fork 41
Add cloud logging service feature #1847
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Yavor16
wants to merge
9
commits into
master
Choose a base branch
from
export-logs-to-logging-service-squash
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
7220532
Add cloud logging service feature
Yavor16 bb7fc04
fix rebase problems
Yavor16 4a2a63a
Fix a bug
Yavor16 a42c170
Add retry
Yavor16 9558829
Fix comments
Yavor16 7c9bf29
Fix comments from Ivas
Yavor16 8e157bb
Fix comments
Yavor16 f1ffc33
Fix unit tests and undeploy bug
Yavor16 8de43a6
Fix comments from Ivan
Yavor16 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14 changes: 14 additions & 0 deletions
14
...ndry/multiapps/controller/core/auditlogging/CloudLoggingServiceConfigurationAuditLog.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| package org.cloudfoundry.multiapps.controller.core.auditlogging; | ||
|
|
||
| import org.cloudfoundry.multiapps.controller.persistence.model.LoggingConfiguration; | ||
|
|
||
| public interface CloudLoggingServiceConfigurationAuditLog { | ||
|
|
||
| void logCreateLoggingConfiguration(String username, String spaceId, LoggingConfiguration loggingConfiguration); | ||
|
|
||
|
|
||
| void logUpdateLoggingConfiguration(String username, String spaceId, LoggingConfiguration newConfiguration); | ||
|
|
||
|
|
||
| void logDeleteLoggingConfiguration(String username, String spaceId, LoggingConfiguration loggingConfiguration); | ||
|
|
||
|
|
||
| void logGetLoggingConfiguration(String username, String spaceId, LoggingConfiguration loggingConfiguration); | ||
|
|
||
| } | ||
52 changes: 52 additions & 0 deletions
52
...ps/controller/core/auditlogging/impl/DefaultCloudLoggingServiceConfigurationAuditLog.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| package org.cloudfoundry.multiapps.controller.core.auditlogging.impl; | ||
|
|
||
| import org.cloudfoundry.multiapps.controller.core.Messages; | ||
| import org.cloudfoundry.multiapps.controller.core.auditlogging.AuditLoggingFacade; | ||
| import org.cloudfoundry.multiapps.controller.core.auditlogging.CloudLoggingServiceConfigurationAuditLog; | ||
| import org.cloudfoundry.multiapps.controller.core.auditlogging.model.AuditLogConfiguration; | ||
| import org.cloudfoundry.multiapps.controller.core.auditlogging.model.ConfigurationChangeActions; | ||
| import org.cloudfoundry.multiapps.controller.persistence.model.LoggingConfiguration; | ||
|
|
||
| import static org.apache.commons.lang3.StringUtils.EMPTY; | ||
|
|
||
| public class DefaultCloudLoggingServiceConfigurationAuditLog implements CloudLoggingServiceConfigurationAuditLog { | ||
|
|
||
| private final AuditLoggingFacade auditLoggingFacade; | ||
|
|
||
| public DefaultCloudLoggingServiceConfigurationAuditLog(AuditLoggingFacade auditLoggingFacade) { | ||
| this.auditLoggingFacade = auditLoggingFacade; | ||
| } | ||
|
|
||
| @Override | ||
| public void logCreateLoggingConfiguration(String username, String spaceId, LoggingConfiguration loggingConfiguration) { | ||
| auditLoggingFacade.logConfigurationChangeAuditLog( | ||
| createAuditLogConfiguration(Messages.LOGGING_CONFIGURATION_CREATE_AUDIT_LOG_CONFIG), | ||
| ConfigurationChangeActions.CONFIGURATION_CREATE); | ||
| } | ||
|
|
||
| @Override | ||
| public void logUpdateLoggingConfiguration(String username, String spaceId, LoggingConfiguration newConfiguration) { | ||
| auditLoggingFacade.logConfigurationChangeAuditLog( | ||
| createAuditLogConfiguration(Messages.LOGGING_CONFIGURATION_UPDATE_AUDIT_LOG_CONFIG), | ||
| ConfigurationChangeActions.CONFIGURATION_UPDATE); | ||
| } | ||
|
|
||
| @Override | ||
| public void logDeleteLoggingConfiguration(String username, String spaceId, LoggingConfiguration loggingConfiguration) { | ||
| auditLoggingFacade.logConfigurationChangeAuditLog( | ||
| createAuditLogConfiguration(Messages.LOGGING_CONFIGURATION_DELETE_AUDIT_LOG_CONFIG), | ||
| ConfigurationChangeActions.CONFIGURATION_DELETE); | ||
| } | ||
|
|
||
| @Override | ||
| public void logGetLoggingConfiguration(String username, String spaceId, LoggingConfiguration loggingConfiguration) { | ||
| auditLoggingFacade.logDataAccessAuditLog(createAuditLogConfiguration(Messages.LOGGING_CONFIGURATION_GET_AUDIT_LOG_CONFIG)); | ||
| } | ||
|
|
||
| private AuditLogConfiguration createAuditLogConfiguration(String message) { | ||
| return new AuditLogConfiguration(EMPTY, | ||
| EMPTY, | ||
| message, | ||
| Messages.LOGGING_CONFIGURATION_GET_AUDIT_LOG_CONFIG); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
154 changes: 154 additions & 0 deletions
154
...va/org/cloudfoundry/multiapps/controller/core/cloudlogging/CloudLoggingServiceClient.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| package org.cloudfoundry.multiapps.controller.core.cloudlogging; | ||
|
|
||
| import java.io.IOException; | ||
| import java.text.MessageFormat; | ||
| import java.time.Duration; | ||
| import java.util.List; | ||
| import java.util.Set; | ||
| import java.util.concurrent.TimeoutException; | ||
|
|
||
| import jakarta.inject.Inject; | ||
| import jakarta.inject.Named; | ||
| import org.cloudfoundry.multiapps.common.util.JsonUtil; | ||
| import org.cloudfoundry.multiapps.controller.persistence.Messages; | ||
| import org.cloudfoundry.multiapps.controller.persistence.model.ExternalOperationLogEntry; | ||
| import org.cloudfoundry.multiapps.controller.persistence.model.LoggingConfiguration; | ||
| import org.cloudfoundry.multiapps.controller.persistence.util.CloudLoggingServiceUtil; | ||
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
| import org.springframework.http.HttpHeaders; | ||
| import org.springframework.http.ResponseEntity; | ||
| import org.springframework.web.reactive.function.client.WebClient; | ||
| import org.springframework.web.reactive.function.client.WebClientException; | ||
| import org.springframework.web.reactive.function.client.WebClientRequestException; | ||
| import org.springframework.web.reactive.function.client.WebClientResponseException; | ||
| import reactor.core.Exceptions; | ||
| import reactor.util.retry.Retry; | ||
|
|
||
| import static org.springframework.http.HttpHeaders.CONTENT_TYPE; | ||
| import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE; | ||
|
|
||
| @Named("cloudLoggingServiceClient") | ||
| public class CloudLoggingServiceClient { | ||
|
|
||
| private static final Logger LOGGER = LoggerFactory.getLogger(CloudLoggingServiceClient.class); | ||
| private static final int MAX_RETRY_ATTEMPTS = 4; | ||
| private static final Duration INITIAL_RETRY_BACKOFF = Duration.ofMillis(500); | ||
| private static final Duration MAX_RETRY_BACKOFF = Duration.ofSeconds(10); | ||
| private static final Duration REQUEST_TIMEOUT = Duration.ofSeconds(30); | ||
| private static final Set<Integer> RETRYABLE_STATUS_CODES = Set.of(408, 425, 429, 500, 502, 503, 504); | ||
|
|
||
| private final CloudLoggingServiceWebClientFactory webClientFactory; | ||
| private final CloudLoggingServiceWebClientCache webClientCache; | ||
| private Retry retrySpec; | ||
|
|
||
| public CloudLoggingServiceClient() { | ||
| this(new DefaultCloudLoggingServiceWebClientFactory(), | ||
| new CloudLoggingServiceWebClientCache()); | ||
| } | ||
|
|
||
| @Inject | ||
| public CloudLoggingServiceClient(CloudLoggingServiceWebClientFactory webClientFactory, | ||
| CloudLoggingServiceWebClientCache webClientCache) { | ||
| this.webClientFactory = webClientFactory; | ||
| this.webClientCache = webClientCache; | ||
| } | ||
|
|
||
| public void withRetrySpec(Retry retrySpec) { | ||
| this.retrySpec = retrySpec; | ||
| } | ||
|
|
||
| public void removeClientFromCache(String operationId) { | ||
| webClientCache.remove(operationId); | ||
| } | ||
|
|
||
| public void sendLogsToCloudLoggingService(LoggingConfiguration loggingConfiguration, | ||
| List<ExternalOperationLogEntry> logEntryBatch) { | ||
| WebClient webClient; | ||
| try { | ||
| webClient = webClientCache.getOrCreate(loggingConfiguration, webClientFactory::createWebClientWithMtls); | ||
| } catch (Exception e) { | ||
| handleSendLogFailure(loggingConfiguration, e); | ||
| return; | ||
| } | ||
| try { | ||
| ResponseEntity<Void> response = executeSendLogHttpRequest(webClient, logEntryBatch); | ||
| if (hasRequestFailed(response)) { | ||
| CloudLoggingServiceUtil.logErrorOrThrowExceptionBasedOnFailSafe(loggingConfiguration, LOGGER, | ||
| Messages.FAILED_TO_SEND_LOG_MESSAGE_TO_CLS); | ||
| } | ||
| } catch (IllegalStateException | WebClientResponseException e) { | ||
| handleSendLogFailure(loggingConfiguration, e); | ||
| } catch (WebClientException e) { | ||
| if (isTransportFailure(Exceptions.unwrap(e))) { | ||
| handleSendLogFailure(loggingConfiguration, e); | ||
| } else { | ||
| throw e; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| private boolean isTransportFailure(Throwable throwable) { | ||
| return throwable instanceof IOException || throwable instanceof TimeoutException; | ||
| } | ||
|
|
||
| private void handleSendLogFailure(LoggingConfiguration loggingConfiguration, Throwable failure) { | ||
| CloudLoggingServiceUtil.logErrorOrThrowExceptionBasedOnFailSafe(loggingConfiguration, LOGGER, | ||
| Messages.FAILED_TO_SEND_LOG_MESSAGE_TO_CLS + ": " | ||
| + describeFailure(failure)); | ||
| } | ||
|
|
||
| private ResponseEntity<Void> executeSendLogHttpRequest(WebClient webClient, List<ExternalOperationLogEntry> logEntryBatch) { | ||
|
Yavor16 marked this conversation as resolved.
|
||
| return webClient.post() | ||
| .header(CONTENT_TYPE, APPLICATION_JSON_VALUE) | ||
| .bodyValue(JsonUtil.toJson(logEntryBatch)) | ||
| .retrieve() | ||
| .toBodilessEntity() | ||
| .timeout(REQUEST_TIMEOUT) | ||
| .retryWhen(retrySpec != null ? retrySpec : buildRetrySpec()) | ||
| .block(); | ||
| } | ||
|
|
||
| private Retry buildRetrySpec() { | ||
| return Retry.backoff(MAX_RETRY_ATTEMPTS, INITIAL_RETRY_BACKOFF) | ||
| .maxBackoff(MAX_RETRY_BACKOFF) | ||
| .jitter(0.5d) | ||
| .filter(this::isRetryableError) | ||
| .doBeforeRetry(retrySignal -> LOGGER.warn(MessageFormat.format(Messages.RETRYING_SEND_LOGS_TO_CLS, | ||
| describeFailure(retrySignal.failure())))) | ||
| .onRetryExhaustedThrow((spec, retrySignal) -> retrySignal.failure()); | ||
| } | ||
|
|
||
| boolean isRetryableError(Throwable throwable) { | ||
| if (throwable instanceof WebClientResponseException responseException) { | ||
| return RETRYABLE_STATUS_CODES.contains(responseException.getStatusCode() | ||
| .value()); | ||
| } | ||
| if (throwable instanceof WebClientRequestException) { | ||
| return true; | ||
| } | ||
| return throwable instanceof IOException; | ||
| } | ||
|
|
||
| private String describeFailure(Throwable throwable) { | ||
| if (throwable instanceof WebClientResponseException responseException) { | ||
| String retryAfter = responseException.getHeaders() | ||
| .getFirst(HttpHeaders.RETRY_AFTER); | ||
| return MessageFormat.format("HTTP {0} {1}{2}", responseException.getStatusCode() | ||
| .value(), | ||
| responseException.getStatusText(), | ||
| retryAfter != null ? " (Retry-After=" + retryAfter + ")" : ""); | ||
| } | ||
| return throwable.getClass() | ||
| .getSimpleName() + ": " + throwable.getMessage(); | ||
| } | ||
|
|
||
| private boolean hasRequestFailed(ResponseEntity<Void> response) { | ||
| if (response == null) { | ||
| return true; | ||
| } | ||
| int statusCode = response.getStatusCode() | ||
| .value(); | ||
| return statusCode < 200 || statusCode > 299; | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it will be better if we move this class in the internal project (to be extended in the internal project)