prysm-pulse/io/logs/logutil_test.go

64 lines
2.0 KiB
Go
Raw Permalink Normal View History

package logs
Mask credentials for logging (#8429) * Added MaskCredentialsLogging to logutil, which masks the user info, path and query. It leaves the hostname and port untouched . Making it more secure during logging * Added MaskCredentialsLogging to logutil, which masks the user info,path and query. It leaves the hostname and port untouched . Making it more secure during logging * Added newline based on the PR checks * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Gazelle * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * added unit tests * updated one test case * added logutil_test.go unit test cases * Refactor validator subnet subscriptions to be non-blocking (#8319) * Use response.NextEpochDuties for aggregator subnet subscriptions (credit: @KaanKC PR #8204). Make committee subnet subscriptions method non-blocking call * Fix test * Fix test Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> * Add Ability to Specify All Public Keys When Exiting Validators (#8399) * add programmatic voluntary exit * add exit all flag * test * lint * add multiple exits test * fix test Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Increase Validation Queue (#8431) Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Validator: add a DEBUG log to show batch attestation save duration (#8432) * Add a debug log to show duration * Autofix issues in 1 file Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * Add Mutex and Block Profiling (#8435) * Implement GetStateRoot in the beacon API (#8402) * update ethereumapis dependency * span * initial implementation * introduce stategen Service interface and MockService * Include AddStateForSlot function in the mock service * return states from mock * add GenesisState to POWChain mock * populate roots in helper state * initialize Slot when creating helper state * tests * code refactor - extract helper functions * gzl * use SetSlot in tests * handle SetSlot error * use new testutil's NewBeaconState * gzl * go mod tidy * rename Service to StateManager * move regex check to helper Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * On Block Cleanup (#8438) * Beacon API: update GetStateRoot (#8437) * Address various feedbacks * Gaz * More nil check * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> Co-authored-by: Radosław Kapka <rkapka@wp.pl> * qualifying my unix user ahaddad - no real changes to the files * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-02-15 21:54:19 +00:00
import (
"fmt"
"os"
Mask credentials for logging (#8429) * Added MaskCredentialsLogging to logutil, which masks the user info, path and query. It leaves the hostname and port untouched . Making it more secure during logging * Added MaskCredentialsLogging to logutil, which masks the user info,path and query. It leaves the hostname and port untouched . Making it more secure during logging * Added newline based on the PR checks * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Gazelle * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * added unit tests * updated one test case * added logutil_test.go unit test cases * Refactor validator subnet subscriptions to be non-blocking (#8319) * Use response.NextEpochDuties for aggregator subnet subscriptions (credit: @KaanKC PR #8204). Make committee subnet subscriptions method non-blocking call * Fix test * Fix test Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> * Add Ability to Specify All Public Keys When Exiting Validators (#8399) * add programmatic voluntary exit * add exit all flag * test * lint * add multiple exits test * fix test Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Increase Validation Queue (#8431) Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Validator: add a DEBUG log to show batch attestation save duration (#8432) * Add a debug log to show duration * Autofix issues in 1 file Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * Add Mutex and Block Profiling (#8435) * Implement GetStateRoot in the beacon API (#8402) * update ethereumapis dependency * span * initial implementation * introduce stategen Service interface and MockService * Include AddStateForSlot function in the mock service * return states from mock * add GenesisState to POWChain mock * populate roots in helper state * initialize Slot when creating helper state * tests * code refactor - extract helper functions * gzl * use SetSlot in tests * handle SetSlot error * use new testutil's NewBeaconState * gzl * go mod tidy * rename Service to StateManager * move regex check to helper Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * On Block Cleanup (#8438) * Beacon API: update GetStateRoot (#8437) * Address various feedbacks * Gaz * More nil check * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> Co-authored-by: Radosław Kapka <rkapka@wp.pl> * qualifying my unix user ahaddad - no real changes to the files * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-02-15 21:54:19 +00:00
"testing"
"github.com/prysmaticlabs/prysm/v5/testing/require"
Mask credentials for logging (#8429) * Added MaskCredentialsLogging to logutil, which masks the user info, path and query. It leaves the hostname and port untouched . Making it more secure during logging * Added MaskCredentialsLogging to logutil, which masks the user info,path and query. It leaves the hostname and port untouched . Making it more secure during logging * Added newline based on the PR checks * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Gazelle * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * added unit tests * updated one test case * added logutil_test.go unit test cases * Refactor validator subnet subscriptions to be non-blocking (#8319) * Use response.NextEpochDuties for aggregator subnet subscriptions (credit: @KaanKC PR #8204). Make committee subnet subscriptions method non-blocking call * Fix test * Fix test Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> * Add Ability to Specify All Public Keys When Exiting Validators (#8399) * add programmatic voluntary exit * add exit all flag * test * lint * add multiple exits test * fix test Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Increase Validation Queue (#8431) Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * Validator: add a DEBUG log to show batch attestation save duration (#8432) * Add a debug log to show duration * Autofix issues in 1 file Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> * Add Mutex and Block Profiling (#8435) * Implement GetStateRoot in the beacon API (#8402) * update ethereumapis dependency * span * initial implementation * introduce stategen Service interface and MockService * Include AddStateForSlot function in the mock service * return states from mock * add GenesisState to POWChain mock * populate roots in helper state * initialize Slot when creating helper state * tests * code refactor - extract helper functions * gzl * use SetSlot in tests * handle SetSlot error * use new testutil's NewBeaconState * gzl * go mod tidy * rename Service to StateManager * move regex check to helper Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> * On Block Cleanup (#8438) * Beacon API: update GetStateRoot (#8437) * Address various feedbacks * Gaz * More nil check * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> * Update beacon-chain/rpc/beaconv1/state_test.go Co-authored-by: Radosław Kapka <rkapka@wp.pl> Co-authored-by: Radosław Kapka <rkapka@wp.pl> * qualifying my unix user ahaddad - no real changes to the files * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go * Update shared/logutil/logutil.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-02-15 21:54:19 +00:00
)
var urltests = []struct {
url string
maskedUrl string
}{
{"https://a:b@xyz.net", "https://***@xyz.net"},
{"https://eth-goerli.alchemyapi.io/v2/tOZG5mjl3.zl_nZdZTNIBUzsDq62R_dkOtY",
"https://eth-goerli.alchemyapi.io/***"},
{"https://google.com/search?q=golang", "https://google.com/***"},
{"https://user@example.com/foo%2fbar", "https://***@example.com/***"},
{"http://john@example.com/#x/y%2Fz", "http://***@example.com/#***"},
{"https://me:pass@example.com/foo/bar?x=1&y=2", "https://***@example.com/***"},
}
func TestMaskCredentialsLogging(t *testing.T) {
for _, test := range urltests {
require.Equal(t, MaskCredentialsLogging(test.url), test.maskedUrl)
}
}
func TestConfigurePersistantLogging(t *testing.T) {
testParentDir := t.TempDir()
// 1. Test creation of file in an existing parent directory
logFileName := "test.log"
existingDirectory := "test-1-existing-testing-dir"
err := ConfigurePersistentLogging(fmt.Sprintf("%s/%s/%s", testParentDir, existingDirectory, logFileName))
require.NoError(t, err)
// 2. Test creation of file along with parent directory
nonExistingDirectory := "test-2-non-existing-testing-dir"
err = ConfigurePersistentLogging(fmt.Sprintf("%s/%s/%s", testParentDir, nonExistingDirectory, logFileName))
require.NoError(t, err)
// 3. Test creation of file in an existing parent directory with a non-existing sub-directory
existingDirectory = "test-3-existing-testing-dir"
nonExistingSubDirectory := "test-3-non-existing-sub-dir"
err = os.Mkdir(fmt.Sprintf("%s/%s", testParentDir, existingDirectory), 0700)
if err != nil {
return
}
err = ConfigurePersistentLogging(fmt.Sprintf("%s/%s/%s/%s", testParentDir, existingDirectory, nonExistingSubDirectory, logFileName))
require.NoError(t, err)
//4. Create log file in a directory without 700 permissions
existingDirectory = "test-4-existing-testing-dir"
err = os.Mkdir(fmt.Sprintf("%s/%s", testParentDir, existingDirectory), 0750)
if err != nil {
return
}
}