> ## Documentation Index
> Fetch the complete documentation index at: https://www.diadata.org/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# DIA Test‐Space with Docker Compose

> Developing, Testing, and Building the DIA platform with Docker Compose

## Prepare

```js theme={"system"}
# Start data store services
(
  cd deployments/local/exchange-scraper
  docker compose -f docker-compose.yml up --build --force-recreate -d
)

# Define env variables needed
export USE_ENV=true
export INFLUXURL=http://localhost:8086
export INFLUXUSER=test
export INFLUXPASSWORD=testtest
export POSTGRES_USER=postgres
export POSTGRES_PASSWORD=password
export POSTGRES_HOST=localhost
export POSTGRES_DB=postgres
export REDISURL=localhost:6379

# Set config directory
export DIA_CONFIG_DIR=./config

# Set local directory as Go dependency
go mod edit -replace "github.com/diadata-org/diadata=$(pwd)" cmd/exchange-scrapers/collector/go.mod
```

## Scrapers

### CEX (Kraken as example)

#### Prepare

```js theme={"system"}
# Initialize the exchange and blockchain metadata
go run ./cmd/services/blockchainservice/main.go

# Prepare pairs metadata
go run ./cmd/services/pairDiscoveryService/main.go -exchange=Kraken -mode=verification
```

#### Run the scraper

```js theme={"system"}
go run ./cmd/exchange-scrapers/collector/collector.go -exchange=Kraken -mode=storeTrades
```

### DEX (CurveFi as example)

#### Prepare

```js theme={"system"}
# Set needed RPC variables
export ETHEREUM_URI_REST=https://rpc.ankr.com/eth
export ETHEREUM_URI_WS=wss://eth-mainnet.g.alchemy.com/v2/7X9hL_lpF-Utavuv1aT49SWkWu9oYZTq

# Prepare the assets metadata
go run ./cmd/assetCollectionService/main.go -source=assetlists -secret=eth_assets -caching=true
go run ./cmd/assetCollectionService/main.go -source=assetlists -secret=non_eth_assets -caching=true
go run ./cmd/assetCollectionService/main.go -source=assetlists -secret=fiat_assets -caching=true
go run ./cmd/assetCollectionService/main.go -source=Curvefi -caching=true

# Prepare and populate pools
go run ./cmd/liquidityScraper/main.go -exchange=Curvefi
```

#### Run the scraper

```js theme={"system"}
go run ./cmd/exchange-scrapers/collector/collector.go -exchange=Curvefi -mode=storeTrades
```

## Clean

```js theme={"system"}
unset USE_ENV
unset INFLUXURL
unset INFLUXUSER
unset INFLUXPASSWORD
unset POSTGRES_USER
unset POSTGRES_PASSWORD
unset POSTGRES_HOST
unset POSTGRES_DB
unset REDISURL
unset DIA_CONFIG_DIR
unset ETHEREUM_URI_REST
unset ETHEREUM_URI_WS
```

### Advanced usage

#### Access services between Docker Composes

On prepare step if you want to connect services between other Compose files, create a `deployments/local/exchange-scraper/docker-compose.override.yml` file:

```js theme={"system"}
version: "3.3"

services:
  influxdb:
    networks:
      - dia-bridge

networks:
  dia-bridge:
    driver: bridge
```

And instead of run the services using this command:

```js theme={"system"}
# Start data store services
(
  cd deployments/local/exchange-scraper
  docker compose -f docker-compose.yml -f docker-compose.override.yml up --build --force-recreate -d
)
```

Now in other Compose file just use like this:

```yaml theme={"system"}
version: "3"

networks:
  exchange-scraper_dia-bridge:
    external: true

services:
  service-x:
    image: service-x-image
    networks:
      - exchange-scraper_dia-bridge
```

Now in Docker context, `service-x` can resolve the `http://influxdb:8086` hostname