Solving Let's Encrypt Challenges with SCP for Limited Web Hosting

The problem: limited hosting without API access

If you’ve ever tried to set up SSL certificates with Let’s Encrypt on a domain hosting provider that doesn’t offer API access, you know the struggle. My situation:

1. My domain is hosted with a provider that doesn’t offer any API for automated certificate management
2. My web space is limited, making it difficult to run a full ACME client directly on the server
3. I needed to automate the certificate renewal process to avoid manual intervention every 90 days

The standard approach for Let’s Encrypt validation is to place a specific challenge file in the .well-known/acme-challenge/ directory of your website. But how do you do this when you can’t run software directly on your hosting provider?

Lego-SCP-Solver

To solve this, I wrote a Go tool that uses the lego ACME client library with a custom challenge solver that uploads the challenge files to my web server over SCP.

The tool works by:

1. Connecting to your web server via SSH
2. Creating the required .well-known/acme-challenge/ directory
3. Uploading the challenge token file via SCP
4. Verifying the file is accessible via HTTP
5. Cleaning up after the challenge is complete

That way I can run certificate issuance from my local machine or a CI/CD pipeline, without needing to install anything on the web server itself.

How it works

The core of the solution is a custom HTTP-01 challenge provider that implements the lego challenge interface. Here’s a simplified version of how it works:

// Present implements the challenge.Provider interface
func (s *SCPSolver) Present(domain, token, keyAuth string) error {
    // Connect to SSH
    if err := s.connect(); err != nil {
        return fmt.Errorf("SSH connection failed: %w", err)
    }
    defer s.sshClient.Close()

    // Create remote directory
    remotePath := s.webrootPath + "/.well-known/acme-challenge"
    if err := s.createRemoteDir(remotePath); err != nil {
        return fmt.Errorf("failed to create remote directory: %w", err)
    }

    // Upload challenge file
    remoteFile := remotePath + "/" + token
    if err := s.uploadFile(remoteFile, keyAuth); err != nil {
        return fmt.Errorf("failed to upload challenge file: %w", err)
    }

    // Set proper permissions for web server access
    if err := s.setPermissions(remotePath, remoteFile); err != nil {
        log.Printf("Warning: Failed to set permissions: %v", err)
    }

    return nil
}

When Let’s Encrypt needs to validate domain ownership, this code:

1. Establishes an SSH connection to the web server
2. Creates the challenge directory if it doesn’t exist
3. Uploads the challenge token with the correct content
4. Sets appropriate permissions so the web server can serve the file

After validation, a similar cleanup function removes the challenge file.

Setting up and using the tool

Using the tool is straightforward. You can either set environment variables:

export LEGO_SCP_HOST="your-server.com"
export LEGO_SCP_USER="your-username"
export LEGO_SCP_KEY_PATH="/path/to/your/ssh/private/key"
export LEGO_SCP_WEBROOT_PATH="/var/www/html"
export LEGO_SCP_EMAIL="your-email@example.com"
export LEGO_SCP_DOMAINS="example.com,www.example.com"
export LEGO_SCP_ACCOUNT_KEY="/path/to/account.key"
export LEGO_SCP_CERT_PATH="/path/to/certificates"

Or use command-line flags:

lego-scp-solver -e your-email@example.com -d example.com,www.example.com \
  --scp-host your-server.com --scp-user username --scp-key ~/.ssh/id_rsa \
  --scp-webroot /var/www/html --cert-path ./certificates

Automating with GitHub Actions

To fully automate the certificate renewal process, I set up a GitHub Actions workflow that runs the tool on a schedule:

name: Renew SSL Certificates

on:
  schedule:
    - cron: "0 0 1 * *" # Run on the 1st of every month
  workflow_dispatch: # Allow manual triggering

jobs:
  renew:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: "1.21"

      - name: Setup SSH key
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa

      - name: Run certificate renewal
        env:
          LEGO_SCP_HOST: ${{ secrets.SCP_HOST }}
          LEGO_SCP_USER: ${{ secrets.SCP_USER }}
          LEGO_SCP_KEY_PATH: ~/.ssh/id_rsa
          LEGO_SCP_WEBROOT_PATH: ${{ secrets.WEBROOT_PATH }}
          LEGO_SCP_EMAIL: ${{ secrets.ACME_EMAIL }}
          LEGO_SCP_DOMAINS: ${{ secrets.DOMAINS }}
          LEGO_SCP_ACCOUNT_KEY: account.key
        run: |
          go run .

      - name: Upload certificates
        uses: actions/upload-artifact@v3
        with:
          name: certificates
          path: ./*.crt

This workflow keeps all the sensitive values in GitHub Secrets and renews the certificates on its own.

Why SCP

The server only needs SSH/SCP access, which nearly every host provides. Once the workflow is in place, certificates renew on schedule without any manual step. File transfers use SSH key authentication, it handles multiple domains and subdomains, and it has almost no dependencies.

The lego ACME library

Lego is a Let’s Encrypt client and ACME library written in Go. It handles obtaining, renewing, and revoking certificates from Let’s Encrypt and other ACME-compatible CAs.

It supports HTTP-01, DNS-01, and TLS-ALPN-01, and ships with built-in providers for many DNS services and hosting platforms. It also lets you plug in a custom challenge solver, which is what this tool does.

The easier alternative: a supported DNS provider

The SCP approach fits my constraints, but if you can choose your DNS provider there’s a simpler path. Lego has built-in support for over 150 DNS providers.

With DNS-01 validation through a supported provider, you can issue wildcard certificates (not possible with HTTP validation), validate domains without exposing your web server to the internet, automate the whole process without custom code, and issue certificates even when port 80 is blocked.

When using DNS-01 validation, Lego creates a TXT record at _acme-challenge.yourdomain.com that Let’s Encrypt verifies. You can check this record yourself using nslookup:

$ nslookup -q=TXT _acme-challenge.yourdomain.com 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Non-authoritative answer:
_acme-challenge.yourdomain.com	text = "IyxgKAO2vD-GRuMQgJfDKI8zcJRZwjTkYOv_xgAQmq4"

Authoritative answers can be found from:
yourdomain.com	nameserver = ns1.example-dns.com.
yourdomain.com	nameserver = ns2.example-dns.com.

This TXT record contains the validation token that proves you control the domain.

If you’re starting a new project or can migrate your DNS, consider using one of these supported providers:

• AWS Route 53
• Cloudflare
• DigitalOcean
• Google Cloud DNS
• Azure DNS
• OVH