# Golang Testing Mocking Strategies
## Summary
This resource outlines strategies for testing private interactions between components in Go while maintaining proper package boundaries. The recommended approach involves making internal functionality public through interfaces, defining these interfaces in the `types` directory, using dependency injection to provide real implementations in production and mocks in tests, and documenting clearly as "internal" APIs to discourage improper usage.
Key benefits:
- Maintains testability across package boundaries
- Preserves conceptual encapsulation with clear documentation
- Follows Go community practices and standard library patterns
- Provides clean separation between production and test code
## Details
### Problem Statement
Testing components in Go that depend on private (unexported) methods from other components while maintaining the following constraints:
- Only exported (public) interfaces should be tested
- Tests should be in their own package (`<package>_tests`) to enforce testing only public APIs
- Mocks should be in their own mock package (`<package>_mocks`)
- Mocks should live in a `mocks` directory
- Mocks should be in files named after the interface they're mocking (`<interface>_mock.go`)
- The main struct in a file should live in the file (e.g., `Client` in `client.go`)
- Other interfaces and types should live in a `types` directory
The specific issue arises when two components in a package (e.g., `RedditClient` and `RedditFetcher`) have a dependency relationship where one uses private methods of the other. Testing the dependent component in isolation requires mocking these private methods, which becomes challenging due to Go's visibility rules across package boundaries.
### Options Considered
#### Option 1: Extract a Public Interface
Create a public interface in the `types` directory that captures the dependencies needed by the dependent component.
**Example:**
```go
// reddit/types/client.go
package reddit
type ClientInterface interface {
FetchRawPosts(query string) ([]RawPost, error)
}
// reddit/client.go
package reddit
type RedditClient struct {
httpClient *http.Client
}
// Public method implementing the interface
func (c *RedditClient) FetchRawPosts(query string) ([]RawPost, error) {
// Implementation that was previously private
return c.doRequest(query)
}
// Still keep private helper methods
func (c *RedditClient) doRequest(query string) ([]RawPost, error) {
// Implementation details
}
// reddit/fetcher.go
package reddit
type RedditFetcher struct {
client ClientInterface
}
func NewRedditFetcher(client ClientInterface) *RedditFetcher {
return &RedditFetcher{client: client}
}
// Client can be mocked in tests
func (f *RedditFetcher) FetchPosts(query string) ([]ProcessedPost, error) {
rawPosts, err := f.client.FetchRawPosts(query)
if err != nil {
return nil, err
}
// Process the raw posts
return processRawPosts(rawPosts), nil
}
// reddit_mocks/client_mock.go
package reddit_mocks
type MockClient struct {
mock.Mock
}
func (m *MockClient) FetchRawPosts(query string) ([]reddit.RawPost, error) {
args := m.Called(query)
return args.Get(0).([]reddit.RawPost), args.Error(1)
}
// reddit_tests/fetcher_test.go
package reddit_tests
func TestFetcher_FetchPosts(t *testing.T) {
mockClient := &reddit_mocks.MockClient{}
mockClient.On("FetchRawPosts", "golang").Return([]reddit.RawPost{
{Title: "Test Post", Body: "Content"},
}, nil)
fetcher := reddit.NewRedditFetcher(mockClient)
posts, err := fetcher.FetchPosts("golang")
assert.NoError(t, err)
assert.Len(t, posts, 1)
mockClient.AssertExpectations(t)
}
```
#### Option 2: Package-level Dependency Injection
Use dependency injection at the package level with factory functions that can be replaced in tests.
**Example:**
```go
// reddit/client.go
package reddit
type redditClient struct {
httpClient *http.Client
}
// Private method - remains private
func (c *redditClient) fetchPostsInternal(query string) ([]RawPost, error) {
// Implementation
}
// Factory function that can be replaced in tests
var newRedditClient = func() *redditClient {
return &redditClient{
httpClient: &http.Client{},
}
}
// reddit/fetcher.go
package reddit
type RedditFetcher struct {
client *redditClient
}
func NewRedditFetcher() *RedditFetcher {
return &RedditFetcher{
client: newRedditClient(),
}
}
func (f *RedditFetcher) FetchPosts(query string) ([]ProcessedPost, error) {
rawPosts, err := f.client.fetchPostsInternal(query)
if err != nil {
return nil, err
}
return processRawPosts(rawPosts), nil
}
// reddit_tests/fetcher_test.go
package reddit_tests
func TestFetcher_FetchPosts(t *testing.T) {
// Save original factory
originalFactory := reddit.newRedditClient
defer func() { reddit.newRedditClient = originalFactory }()
// Mock client
mockClient := &mockRedditClient{}
mockClient.returnPosts = []reddit.RawPost{
{Title: "Test Post", Body: "Content"},
}
// Replace factory function
reddit.newRedditClient = func() *redditClient {
return mockClient
}
fetcher := reddit.NewRedditFetcher()
posts, err := fetcher.FetchPosts("golang")
assert.NoError(t, err)
assert.Len(t, posts, 1)
}
type mockRedditClient struct {
returnPosts []reddit.RawPost
returnError error
}
func (m *mockRedditClient) fetchPostsInternal(query string) ([]reddit.RawPost, error) {
return m.returnPosts, m.returnError
}
```
#### Option 3: Move Client Implementation into Fetcher
Simplify by moving necessary functionality directly into the fetcher if appropriate.
**Example:**
```go
// reddit/fetcher.go
package reddit
type RedditFetcher struct {
httpClient *http.Client
}
func NewRedditFetcher() *RedditFetcher {
return &RedditFetcher{
httpClient: &http.Client{},
}
}
// All functionality moved into fetcher
func (f *RedditFetcher) FetchPosts(query string) ([]ProcessedPost, error) {
// Functionality previously in RedditClient
req, err := http.NewRequest("GET", "https://reddit.com/r/"+query+".json", nil)
if err != nil {
return nil, err
}
resp, err := f.httpClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
var rawPosts []RawPost
// Parse JSON response
return processRawPosts(rawPosts), nil
}
// reddit_tests/fetcher_test.go
package reddit_tests
func TestFetcher_FetchPosts(t *testing.T) {
// Use httptest package to mock HTTP responses
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte(`{"data": {"children": [{"data": {"title": "Test Post"}}]}}`))
}))
defer server.Close()
// Test with the mock HTTP server
fetcher := reddit.NewRedditFetcher()
// Inject custom HTTP client that points to test server
fetcher.httpClient = server.Client()
posts, err := fetcher.FetchPosts("golang")
assert.NoError(t, err)
assert.Len(t, posts, 1)
}
```
#### Option 4: Make Private Methods Public
Make the necessary methods public purely for testing purposes.
**Example:**
```go
// reddit/client.go
package reddit
type RedditClient struct {
httpClient *http.Client
}
// Private method made public for testing
// explicit naming - <FunctionName>Internal
func (c *RedditClient) FetchPostsInternal(query string) ([]RawPost, error) {
// Implementation
}
// reddit/fetcher.go
package reddit
type RedditFetcher struct {
client *RedditClient
}
func NewRedditFetcher(client *RedditClient) *RedditFetcher {
return &RedditFetcher{client: client}
}
func (f *RedditFetcher) FetchPosts(query string) ([]ProcessedPost, error) {
// Using the now-public method that was conceptually private
rawPosts, err := f.client.FetchPostsInternal(query)
if err != nil {
return nil, err
}
return processRawPosts(rawPosts), nil
}
// reddit_mocks/client_mock.go
package reddit_mocks
type MockRedditClient struct {
mock.Mock
}
// Mock the now-public method
func (m *MockRedditClient) FetchPostsInternal(query string) ([]reddit.RawPost, error) {
args := m.Called(query)
return args.Get(0).([]reddit.RawPost), args.Error(1)
}
// reddit_tests/fetcher_test.go
package reddit_tests
func TestFetcher_FetchPosts(t *testing.T) {
mockClient := &reddit_mocks.MockRedditClient{}
mockClient.On("FetchPostsInternal", "golang").Return([]reddit.RawPost{
{Title: "Test Post", Body: "Content"},
}, nil)
fetcher := reddit.NewRedditFetcher(mockClient)
posts, err := fetcher.FetchPosts("golang")
assert.NoError(t, err)
assert.Len(t, posts, 1)
mockClient.AssertExpectations(t)
}
```
### Recommended Solution
The recommended solution is a blend of **Option 1 and Option 4** - essentially making private methods public through interfaces, while adding documentation to clarify usage intentions:
```go
// reddit/types/client.go
package reddit
// ClientInterface provides access to raw Reddit data.
// NOTE: This interface is primarily for internal use by RedditFetcher.
// External consumers should use FetcherInterface instead.
type ClientInterface interface {
// FetchRawPostData retrieves unprocessed post data from Reddit.
// WARNING: This is an internal API. External code should use RedditFetcher.FetchPosts.
FetchPostsInternal(query string) ([]Post, error)
}
// reddit/types/fetcher.go
package reddit
// FetcherInterface is the primary public API for retrieving Reddit posts.
type FetcherInterface interface {
// FetchPosts retrieves fully processed Reddit posts.
FetchPosts(query string) ([]Post, error)
}
// reddit/client.go
package reddit
type RedditClient struct {
httpClient *http.Client
}
// Implementation of the interface method that was conceptually private
// but now exposed for testing purposes
func (c *RedditClient) FetchPostsInternal(query string) ([]Post, error) {
// Implementation that would have been private in an ideal world
}
// reddit/fetcher.go
package reddit
type RedditFetcher struct {
client ClientInterface
}
func NewRedditFetcher(client ClientInterface) *RedditFetcher {
return &RedditFetcher{client: client}
}
// Convenience factory with default implementation
func NewDefaultRedditFetcher() *RedditFetcher {
return NewRedditFetcher(NewRedditClient())
}
func (f *RedditFetcher) FetchPosts(query string) ([]Post, error) {
post, err := f.client.FetchPostsInternal(query)
if err != nil {
return nil, err
}
return processRawData(post), nil
}
```
This solution acknowledges that we're making conceptually private functionality public out of necessity for testing, but we're doing it in a structured way through interfaces with clear documentation.
### Documentation Best Practices
To prevent misuse of internal-facing interfaces:
1. **Package-level documentation**:
```go
/*
Package reddit provides functionality for interacting with Reddit.
Architecture:
This package uses a layered approach:
- RedditClient: Low-level API client (internal use only)
- RedditFetcher: Public API for fetching processed post data
IMPORTANT: External code should only interact with RedditFetcher, not directly with RedditClient.
*/
package reddit
```
1. **Interface documentation**:
```go
// ClientInterface is an INTERNAL interface used by RedditFetcher.
// External code should use FetcherInterface instead.
type ClientInterface interface {
// ...
}
```
1. **Naming conventions**:
```go
// Clearly indicates internal use
type ClientInternal interface {
// ...
}
// Public API
type Fetcher interface {
// ...
}
```
### Examples from Go Standard Library
The Go standard library uses similar patterns in several places:
1. **`net/http`** package: The `http.Client` has a `Transport` field that accepts an `http.RoundTripper` interface, allowing for mocking of HTTP requests in tests:
<https://golang.org/pkg/net/http/#Client>
2. **`database/sql`** package: Uses driver interfaces that allow mocking database connections:
<https://golang.org/pkg/database/sql/driver/>
3. **`io`** package: Defines interfaces like `io.Reader` and `io.Writer` that allow for dependency injection and easier testing:
<https://golang.org/pkg/io/>
4. **`net/http/httptest`**: Provides testing utilities that implement standard interfaces:
<https://golang.org/pkg/net/http/httptest/>
### Additional Resources
1. [Go Testing and Mocking](https://golang.org/doc/tutorial/add-a-test) - Official Go tutorials on testing
2. [Effective Go: Interfaces](https://golang.org/doc/effective_go#interfaces) - Guidance on interface design in Go
3. [Mitchell Hashimoto's Testing Strategies Talk](https://youtu.be/8hQG7QlcLBk) - Good overview of testing patterns in Go
4. [Go Interfaces in Practice](https://about.sourcegraph.com/blog/go/gophercon-2018-how-do-you-structure-your-go-apps) - Practical interface design in Go
5. [Standard Package Layout](https://github.com/golang-standards/project-layout) - Community standards for Go project layout
## 🔗 Related Resources
- [Functional Options Pattern in Go](Functional%20Options%20Pattern%20in%20Go.md) - Another useful pattern for Go programming
- [Test Driven Development (TDD)](Test%20Driven%20Development%20%28TDD%29.md) - Complementary testing methodology
- [SOLID Principles](SOLID%20Principles.md) - Design principles that align with this testing approach