> ## Documentation Index
> Fetch the complete documentation index at: https://developers.gopher-ai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Miners

Miners use the Gopher Worker to scrape data from a secure TEE enclave. It uses the ego framework to build, run and sign the binary.

## Hardware Prerequisites

### Required Hardware

<CardGroup cols={2}>
  <Card title="🔒 Intel SGX Support" icon="microchip">
    * Intel SGX 2.0+ enabled CPU required
    * Hardware-level security enclave
    * Secure computation environment
  </Card>

  <Card title="💻 Compatible Processors" icon="server">
    * 11th Gen+ Intel Core series
    * Modern Intel Xeon processors
    * Must have SGX feature enabled in BIOS
  </Card>
</CardGroup>

<Note>
  Verify your CPU supports Intel SGX 2.0 or higher before setting up mining operations. The SGX feature must also be enabled in your system BIOS.
</Note>

### Cloud Provider Recommendations

For optimal TEE mining performance and reliability, we recommend using Microsoft Azure. This provider has been extensively tested and validated by our team to ensure seamless compatibility with Masa Network's TEE mining requirements:

<CardGroup cols={2}>
  <Card title="Microsoft Azure" icon="microsoft">
    ✓ Offers SGX-enabled virtual machines\
    ✓ Proven reliability for TEE workloads\
    ✓ Validated for Masa Network mining\
    ✓ Our recommended provider
  </Card>

  <Card title="Alibaba Cloud" icon="ban" color="red">
    ✗ Not supported for TEE mining\
    ✗ Uses outdated SGX API (v3)\
    ✗ Incompatible with Masa requirements\
    ✗ Please use Azure or other validated providers
  </Card>
</CardGroup>

### Azure instance capacity planning

When running TEE miners on Azure, it's important to understand the capacity limits to optimize costs and performance. Here's what we've validated:

<CardGroup cols={2}>
  <Card title="Instance Specifications" icon="server">
    Standard DC4s v2

    * 4 vCPUs
    * 16 GiB Memory
    * SGX-enabled
  </Card>

  <Card title="Miner Capacity" icon="chart-line">
    * Maximum 4 TEE miners per instance
    * 1 miner per vCPU recommended
    * Attempting to run >4 miners causes automatic termination
  </Card>
</CardGroup>

<Tip>
  To optimize costs, you can run multiple miners on a single Azure instance as long as you stay within the vCPU limits. This approach is more cost-effective than running individual instances for each miner.
</Tip>

## System Setup & Configuration

### Prerequisites

Before starting TEE mining operations, ensure you have:

<CardGroup cols={2}>
  <Card title="Docker Setup" icon="docker">
    ✓ Docker installed and running\
    ✓ Permissions to run containers
  </Card>

  <Card title="Network Keys" icon="key">
    ✓ BTCLI to installation to manage your wallet\
    ✓ Generate coldkey and hotkey mnemonics and register your miner
  </Card>
</CardGroup>

<AccordionGroup>
  <Accordion icon="code" title="Installing the Bittensor BTCLI">
    <Tabs>
      <Tab title="masOS and Linux">
        To install the BTCLI please follow the [Bittensor BTCLI docs](https://docs.bittensor.com/getting-started/install-btcli).
      </Tab>

      <Tab title="Windows">
        Note that there is limited support for Windows at the moment. We recommend using WSL2 to run the BTCLI.
        Please reference the Bittensor CLI where it covers windows installation  [here](https://docs.bittensor.com/getting-started/install-btcli).
      </Tab>
    </Tabs>
  </Accordion>
</AccordionGroup>

<AccordionGroup>
  <Accordion icon="code" title="Installing Docker">
    <Tabs>
      <Tab title="Linux">
        To install Docker, follow these steps:

        **Linux:**

        ```bash theme={null}
        # Install Docker using snap
        sudo snap install docker
        ```

        ```bash theme={null}
        # Add user to docker group and set permissions
        sudo usermod -aG docker $USER
        ```

        After running these commands, restart your system for the changes to take effect.

        Verify installation by running:

        ```bash theme={null}
        docker --version
        ```
      </Tab>

      <Tab title="Windows">
        For windows users, we recommend using WSL2 to run Docker. Please reference the Docker docs where it covers windows installation  [here](https://docs.docker.com/desktop/install/windows-install/).
        Then use the linux instructions above to install docker.
      </Tab>
    </Tabs>
  </Accordion>
</AccordionGroup>

<AccordionGroup>
  <Accordion icon="code" title="Generate coldkey and hotkey mnemonics and register your miner">
    ### Creating your wallet mnemonics and register your miner

    Follow these steps to create your wallet keys and register as a miner on the network. You'll need to generate both a coldkey (for secure storage) and hotkey (for active mining operations).

    <Steps>
      <Step title="Create a Cold Wallet">
        First, create a cold wallet for your `miner`:

        <CodeBlock language="bash">
          {`btcli wallet new_coldkey --wallet.name miner`}
        </CodeBlock>
      </Step>

      <Step title="Create a Hot Wallet">
        Next, create a hot wallet for your cold wallet:

        <CodeBlock language="bash">
          {`btcli wallet new_hotkey --wallet.name miner --wallet.hotkey default`}
        </CodeBlock>

        <Note>
          Make sure to use the same wallet name (miner) that you used in Step 1.
        </Note>
      </Step>

      <Step title="Verify Your Wallets">
        Finally, verify that both wallets were created successfully:

        <CodeBlock language="bash">
          {`btcli w overview`}
        </CodeBlock>

        You should see your `miner` wallet listed, along with its associated hotkey (`default`).

        <Tip>
          Keep your wallet passwords safe! You'll need them for future operations.
        </Tip>
      </Step>

      <Step title="Register Your Node">
        Register your node as a miner on the subnet:

        ```sh theme={null}
        btcli subnet register --netuid 42
        ```

        <Note>Replace the --netuid with 42 for mainnet or 165 for testnet</Note>
        This command will:

        * Configure your node as a miner
        * Set up the necessary network connections
      </Step>

      <Step title="Verify Registration">
        Check that your miner is properly registered on the subnet:

        ```sh theme={null}
        btcli s metagraph --netuid 165 --network test
        ```

        <Note>Replace the --netuid with 42 for mainnet or 165 for testnet</Note>

        This command will display the subnet metagraph, showing:

        * All registered nodes and their roles
        * Your miner's status and registration
        * Current network topology

        <Note>
          Look for your hotkey address in the output to confirm successful registration. If you don't see your node listed, wait a few minutes and try again.
        </Note>
      </Step>
    </Steps>
  </Accordion>
</AccordionGroup>

<Note>
  Keep your coldkey and hotkey mnemonics secure and backed up. These are required for mining operations and cannot be recovered if lost.
</Note>

## Subnet 42 Miner setup

To begin mining on Subnet 42, follow these steps:

<Steps>
  <Step title="Clone Repository">
    First, clone the Subnet 42 repository which contains the docker compose file and .env example:

    ```sh theme={null}
    git clone https://github.com/gopher-lab/subnet-42.git
    ```
  </Step>

  <Step title="Navigate to Directory">
    Change into the repository directory:

    ```sh theme={null}
    cd subnet-42
    ```
  </Step>

  <Step title="Copy Environment File">
    Create your environment configuration file:

    ```sh theme={null}
    cp .env.example .env
    ```
  </Step>

  <Step title="Edit Environment File">
    Wallet Setup - you can add either your mnemonics, or load your wallet from disk:

    ```bash theme={null}
    COLDKEY_MNEMONIC="your_coldkey_mnemonic"
    HOTKEY_MNEMONIC="your_hotkey_mnemonic"
    ```

    or

    ```bash theme={null}
    WALLET_NAME="your_wallet_name"
    HOTKEY_NAME="your_hotkey_name"
    WALLET_PATH="~/.bittensor"
    ```

    Next, add your role and network configuration:

    ```bash theme={null}
    ROLE="miner"
    NETUID="42" #(165 for testnet)
    SUBTENSOR_NETWORK="finney" #(test for testnet)
    OVERRIDE_EXTERNAL_IP="your_external_ip_address"
    MINER_PORT="your_miner_port"
    LISTEN_ADDRESS=":your_tee_worker_port"
    MINER_TEE_ADDRESS="your_tee_worker_address:<your_tee_worker_port>"
    ```

    Finally, add credentials for your TEE worker:

    ```bash theme={null}
    TWITTER_ACCOUNTS="username:password,username2:password2..." #(to scrape twitter with cookies)
    TWITTER_API_KEYS="key1,key2..." #(to scrape twitter with API keys)
    APIFY_API_KEY="key" #(to support Apify actors)
    CLAUDE_API_KEY="key" #(for web / llm processing)
    ```
  </Step>

  <Step title="Start Your Node">
    Start your node with Docker Compose:

    ```sh theme={null}
    docker compose --profile miner up -d
    ```

    This will:

    * Launch the required containers for your node type
    * Initialize your miner node
    * Initialize your TEE worker
    * Connect to the subnet network
  </Step>

  <Step title="Verify TEE Worker">
    Once your node is running, verify the TEE worker is accessible by visiting your configured address in a browser:

    ```
    https://<your-ip-address>:8080
    ```

    You should see a response indicating the TEE worker is running. If you get a connection error:

    * Verify your IP address is correct in the .env file
    * Check that port 8080 is open in your firewall
    * Ensure the TEE worker container is running properly

    <Note>
      The browser may show a security warning due to the self-signed certificate. This is expected and you can proceed to verify the endpoint is responding.
    </Note>
  </Step>

  <Step title="Monitor Logs">
    Monitor your miner's logs to ensure everything is running properly.

    ```sh theme={null}
    docker logs <container-name>
    ```

    Note that subnet miners run with two containers, one for the neuron (miner) and one for the worker (tee)
  </Step>
</Steps>

## Apify Support

Miners are able to support various Apify actors by setting an `APIFY_API_KEY` in your .env file.  This will give you default access to the Apify API.  Gopher supports the following actors - it is the miner's responsibility to rent the actor if needed!

* `trudax~reddit-scraper`
* `epctex~tiktok-search-scraper`
* `lexis-solutions~tiktok-trending-videos-scraper`
* `dusan.vystrcil~llm-dataset-processor`
* `kaitoeasyapi~premium-x-follower-scraper-following-data`
* `apify~website-content-crawler`

## LLM Support

Some jobs include additional LLM processing, such as `web` `scraper` jobs.  Miners may add either a `CLAUDE_API_KEY` or `GEMINI_API_KEY` to their .env file to support this type of work.

## Cookie Management

If you have `TWITTER_ACCOUNTS` defined in your .env file, you need to provide relevant cookies in the `.masa` folder of your TEE worker.  Each JSON file must be named with the `username` of the account and a `_twitter_cookies.json` suffix.(This will of course soon change to `.gopher`)
Gopher provides helpful tools via [Docker commands](https://github.com/gopher-lab/subnet-42/blob/main/docker-compose.yml), and a Python script built with Chromium [here](https://github.com/gopher-lab/subnet-42/blob/main/scripts/cookie_grabber.py).

## Upgrading your miner

<Steps>
  <Step title="Stop current miner">
    ```sh theme={null}
    docker compose -p <container-name> down
    ```
  </Step>

  <Step title="Start new version">
    ```sh theme={null}
    docker compose --profile miner up -d
    ```

    The latest docker image will automatically be pulled and used, as defined in the docker-compose.yml file.
  </Step>
</Steps>

### Frequently Asked Questions

<AccordionGroup>
  ### ⚙️ Setup & Configuration

  <Accordion title="Can I run multiple miners on the same instance?">
    Yes, multiple miners can run on the same machine. However, each miner must have its own TEE worker. A single TEE worker cannot be shared across multiple miners.
  </Accordion>

  <Accordion title="How many miners can I run per instance?">
    The number of miners per instance depends on the system resources and performance considerations. Each miner needs its own TEE worker, so the setup must account for computational capacity.
  </Accordion>

  <Accordion title="Can I use a single scraper for multiple miners?">
    No, the scraper (worker) is tied to a specific miner's TEE worker. Each miner must have its own dedicated scraping logic within its TEE environment.
  </Accordion>

  <Accordion title="Do I need separate Twitter/X accounts per miner?">
    Yes, but a miner can have multiple Twitter accounts/API keys assigned. It can be X/Twitter accounts you scrape and/or X/Twitter API accounts. We recommend the Twitter API Pro tier for full-archive search. or a combination of tiers for higher stability.
  </Accordion>

  <Accordion title="Can I authenticate via Twitter/X API instead of premium account scraing with logins?">
    Yes, you can authenticate via the X/Twitter API instead of relying on premium account logins, which can be unstable. However, premium accounts remain an option.
  </Accordion>

  <Accordion title="Should I invest in a Twitter API subscription?">
    Yes, it is recommended to consider Twitter/X API Pro or a combination of tiers. This provides better access to full-archive search, improves data reliability, and reduces the risk of scraping failures.
  </Accordion>

  <Accordion title="Will miners be able to share TEE workers in the future?">
    No, when moving to mainnet, miners will not be able to share TEE worker instances. This is possible only in the testnet environment and not recommended.
  </Accordion>

  <Accordion title="How does Twitter/X data access affect performance?">
    Both stable authentication and reliable data access are critical for miner performance and rewards. Your miner's ability to consistently return high-quality Twitter/X data impacts scoring. Poor performance can result from:

    * Authentication failures
    * Slow response times
    * Missing or incomplete data
    * Rate limiting issues
    * API quota exhaustion

    We recommend using the Twitter API with the following tiers:

    * **Basic Tier (\$100/month)**
      * Good starting point for testing
      * Limited to 500k tweets/month
      * Recent search only (7 days)
    * **Pro Tier (\$5000/month)**
      * Full archive search access
      * Up to 1M tweets/month
      * Higher rate limits
      * More reliable for production

    You can use a combination of tiers to achieve the best performance and stability. You also have the option to use premium accounts with logins but can be more unstable..

    A well-optimized setup with reliable API access or stable premium accounts, combined with efficient data retrieval and processing, is key to maintaining high performance scores and maximizing rewards.
  </Accordion>
</AccordionGroup>