DevelopmentSecurity

Solving Let's Encrypt Challenges with SCP for Limited Web Hosting

Updated June 18, 2025
#golang#letsencrypt#acme#ssl#certificates#webhosting

The Problem: Limited Domain Hosting Without API Access

If you’ve ever tried to set up SSL certificates with Let’s Encrypt on a domain hosting provider that doesn’t offer API access, you know the struggle. My situation was particularly challenging:

  1. My domain is hosted with a provider that doesn’t offer any API for automated certificate management
  2. My web space is limited, making it difficult to run a full ACME client directly on the server
  3. I needed to automate the certificate renewal process to avoid manual intervention every 90 days

The standard approach for Let’s Encrypt validation is to place a specific challenge file in the .well-known/acme-challenge/ directory of your website. But how do you do this when you can’t run software directly on your hosting provider?

Enter Lego-SCP-Solver

To solve this problem, I created a Go-based tool that leverages the excellent lego ACME client library with a custom challenge solver that uses SCP (Secure Copy Protocol) to upload the challenge files to my web server.

The tool works by:

  1. Connecting to your web server via SSH
  2. Creating the required .well-known/acme-challenge/ directory
  3. Uploading the challenge token file via SCP
  4. Verifying the file is accessible via HTTP
  5. Cleaning up after the challenge is complete

This approach allows me to run the certificate issuance process from my local machine or a CI/CD pipeline, without needing to install anything on the web server itself.

How It Works

The core of the solution is a custom HTTP-01 challenge provider that implements the lego challenge interface. Here’s a simplified version of how it works:

// Present implements the challenge.Provider interface
func (s *SCPSolver) Present(domain, token, keyAuth string) error {
    // Connect to SSH
    if err := s.connect(); err != nil {
        return fmt.Errorf("SSH connection failed: %w", err)
    }
    defer s.sshClient.Close()

    // Create remote directory
    remotePath := s.webrootPath + "/.well-known/acme-challenge"
    if err := s.createRemoteDir(remotePath); err != nil {
        return fmt.Errorf("failed to create remote directory: %w", err)
    }

    // Upload challenge file
    remoteFile := remotePath + "/" + token
    if err := s.uploadFile(remoteFile, keyAuth); err != nil {
        return fmt.Errorf("failed to upload challenge file: %w", err)
    }

    // Set proper permissions for web server access
    if err := s.setPermissions(remotePath, remoteFile); err != nil {
        log.Printf("Warning: Failed to set permissions: %v", err)
    }

    return nil
}

When Let’s Encrypt needs to validate domain ownership, this code:

  1. Establishes an SSH connection to the web server
  2. Creates the challenge directory if it doesn’t exist
  3. Uploads the challenge token with the correct content
  4. Sets appropriate permissions so the web server can serve the file

After validation, a similar cleanup function removes the challenge file.

Setting Up and Using the Tool

Using the tool is straightforward. You can either set environment variables:

export LEGO_SCP_HOST="your-server.com"
export LEGO_SCP_USER="your-username"
export LEGO_SCP_KEY_PATH="/path/to/your/ssh/private/key"
export LEGO_SCP_WEBROOT_PATH="/var/www/html"
export LEGO_SCP_EMAIL="your-email@example.com"
export LEGO_SCP_DOMAINS="example.com,www.example.com"
export LEGO_SCP_ACCOUNT_KEY="/path/to/account.key"
export LEGO_SCP_CERT_PATH="/path/to/certificates"

Or use command-line flags:

lego-scp-solver -e your-email@example.com -d example.com,www.example.com \
  --scp-host your-server.com --scp-user username --scp-key ~/.ssh/id_rsa \
  --scp-webroot /var/www/html --cert-path ./certificates

Automating with GitHub Actions

To fully automate the certificate renewal process, I set up a GitHub Actions workflow that runs the tool on a schedule:

name: Renew SSL Certificates

on:
  schedule:
    - cron: "0 0 1 * *" # Run on the 1st of every month
  workflow_dispatch: # Allow manual triggering

jobs:
  renew:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: "1.21"

      - name: Setup SSH key
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa

      - name: Run certificate renewal
        env:
          LEGO_SCP_HOST: ${{ secrets.SCP_HOST }}
          LEGO_SCP_USER: ${{ secrets.SCP_USER }}
          LEGO_SCP_KEY_PATH: ~/.ssh/id_rsa
          LEGO_SCP_WEBROOT_PATH: ${{ secrets.WEBROOT_PATH }}
          LEGO_SCP_EMAIL: ${{ secrets.ACME_EMAIL }}
          LEGO_SCP_DOMAINS: ${{ secrets.DOMAINS }}
          LEGO_SCP_ACCOUNT_KEY: account.key
        run: |
          go run .

      - name: Upload certificates
        uses: actions/upload-artifact@v3
        with:
          name: certificates
          path: ./*.crt

This workflow securely stores all sensitive information in GitHub Secrets and runs the certificate renewal process automatically.

Benefits of This Approach

  1. No server-side installation required: Works with any hosting provider that offers SSH/SCP access
  2. Fully automated: Set it and forget it - certificates renew automatically
  3. Secure: Uses SSH key authentication for secure file transfers
  4. Flexible: Works with multiple domains and subdomains
  5. Lightweight: Minimal dependencies and resource usage

The Lego ACME Library: A Powerful Foundation

Lego is a Let’s Encrypt client and ACME library written in Go. It provides a complete solution for obtaining, renewing, and revoking SSL certificates from Let’s Encrypt and other ACME-compatible certificate authorities.

What makes Lego particularly powerful is its extensibility. It supports multiple challenge types (HTTP-01, DNS-01, TLS-ALPN-01) and comes with built-in providers for many popular DNS services and hosting platforms. However, its true strength lies in its ability to be extended with custom challenge solvers.

The Easier Alternative: Using Supported DNS Providers

While my SCP solution works well for my specific constraints, I should mention that there’s an easier path if you have the flexibility to choose your domain registrar or DNS provider. Lego has built-in support for over 150 DNS providers, making certificate issuance much simpler if you use one of these services.

With DNS-01 validation through a supported provider, you can:

  • Issue wildcard certificates (which isn’t possible with HTTP validation)
  • Validate domains without exposing your web server to the internet
  • Automate the entire process without custom code
  • Issue certificates even when port 80 is blocked

When using DNS-01 validation, Lego creates a TXT record at _acme-challenge.yourdomain.com that Let’s Encrypt verifies. You can check this record yourself using nslookup:

$ nslookup -q=TXT _acme-challenge.yourdomain.com 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Non-authoritative answer:
_acme-challenge.yourdomain.com	text = "IyxgKAO2vD-GRuMQgJfDKI8zcJRZwjTkYOv_xgAQmq4"

Authoritative answers can be found from:
yourdomain.com	nameserver = ns1.example-dns.com.
yourdomain.com	nameserver = ns2.example-dns.com.

This TXT record contains the validation token that proves you control the domain.

If you’re starting a new project or can migrate your DNS, consider using one of these supported providers:

  • AWS Route 53
  • Cloudflare
  • DigitalOcean
  • Google Cloud DNS
  • Azure DNS
  • OVH
Back to all posts