-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Added batch operations support for atomic updates
- Implemented safe iterator pattern - Enhanced performance for bulk operations - Added comprehensive benchmarking suite - Improved documentation and examples
- Loading branch information
Showing
10 changed files
with
1,043 additions
and
102 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,182 +1,208 @@ | ||
# ShrinkableMap | ||
|
||
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) | ||
[![Go Reference](https://pkg.go.dev/badge/github.com/jongyunha/shrinkmap.svg)](https://pkg.go.dev/github.com/jongyunha/shrinkmap) | ||
[![Go Report Card](https://goreportcard.com/badge/github.com/jongyunha/shrinkmap)](https://goreportcard.com/report/github.com/jongyunha/shrinkmap) | ||
[![Coverage Status](https://coveralls.io/repos/github/jongyunha/shrinkmap/badge.svg?branch=main)](https://coveralls.io/github/jongyunha/shrinkmap?branch=main) | ||
|
||
ShrinkableMap is a high-performance, generic, thread-safe map implementation for Go that automatically manages memory by shrinking its internal storage when items are deleted. It provides a solution to the common issue where Go's built-in maps don't release memory after deleting elements. | ||
ShrinkableMap is a high-performance, generic, thread-safe map implementation for Go that automatically manages memory by shrinking its internal storage when items are deleted. It addresses the common issue where Go's built-in maps don't release memory after deleting elements. | ||
|
||
## Features | ||
## 🚀 Features | ||
|
||
- 🚀 Generic type support for type-safe operations | ||
- 🔒 Thread-safe implementation with atomic operations | ||
- 📉 Automatic memory shrinking with configurable policies | ||
- ⚙️ Advanced concurrent shrinking behavior | ||
- 📊 Thread-safe performance and error metrics | ||
- 🛡️ Panic recovery and error tracking | ||
- 🔍 Safe state inspection with snapshots | ||
- 🧹 Graceful resource cleanup | ||
- 💪 Production-ready with comprehensive tests | ||
- 🎯 Zero external dependencies | ||
- **Type Safety** | ||
- Generic type support for compile-time type checking | ||
- Type-safe operations for all map interactions | ||
|
||
## Installation | ||
- **Performance** | ||
- Optimized concurrent access with minimal locking | ||
- Efficient atomic operations for high throughput | ||
- Batch operations for improved performance | ||
|
||
- **Memory Management** | ||
- Automatic memory shrinking with configurable policies | ||
- Advanced concurrent shrinking behavior | ||
- Memory-efficient iterators | ||
|
||
- **Reliability** | ||
- Thread-safe implementation | ||
- Panic recovery and error tracking | ||
- Comprehensive metrics collection | ||
|
||
- **Developer Experience** | ||
- Safe iteration with snapshot support | ||
- Batch operations for bulk processing | ||
- Clear error reporting and metrics | ||
- Zero external dependencies | ||
- Production-ready with extensive tests | ||
|
||
## 📦 Installation | ||
|
||
```bash | ||
go get github.com/jongyunha/shrinkmap | ||
``` | ||
|
||
## Quick Start | ||
## 🔧 Quick Start | ||
|
||
```go | ||
package main | ||
|
||
import ( | ||
"fmt" | ||
"time" | ||
"github.com/jongyunha/shrinkmap" | ||
) | ||
|
||
func main() { | ||
// Create a new map with string keys and int values | ||
// Create a new map with default configuration | ||
sm := shrinkmap.New[string, int](shrinkmap.DefaultConfig()) | ||
|
||
// Ensure cleanup when done | ||
defer sm.Stop() | ||
|
||
// Set values | ||
// Basic operations | ||
sm.Set("one", 1) | ||
sm.Set("two", 2) | ||
|
||
// Get value | ||
if value, exists := sm.Get("one"); exists { | ||
fmt.Printf("Value: %d\n", value) | ||
} | ||
|
||
// Delete value | ||
sm.Delete("one") | ||
|
||
// Get current state snapshot | ||
snapshot := sm.Snapshot() | ||
for _, kv := range snapshot { | ||
fmt.Printf("Key: %v, Value: %v\n", kv.Key, kv.Value) | ||
} | ||
|
||
// Get metrics including error statistics | ||
metrics := sm.GetMetrics() | ||
fmt.Printf("Total operations: %d\n", metrics.TotalItemsProcessed()) | ||
fmt.Printf("Total errors: %d\n", metrics.TotalErrors()) | ||
fmt.Printf("Total panics: %d\n", metrics.TotalPanics()) | ||
} | ||
``` | ||
|
||
## Advanced Features | ||
## 💡 Advanced Usage | ||
|
||
### Error Tracking and Recovery | ||
### Batch Operations | ||
|
||
Monitor and track errors with detailed information: | ||
Efficiently process multiple operations atomically: | ||
|
||
```go | ||
metrics := sm.GetMetrics() | ||
|
||
// Get error statistics | ||
totalErrors := metrics.TotalErrors() | ||
totalPanics := metrics.TotalPanics() | ||
lastPanicTime := metrics.LastPanicTime() | ||
|
||
// Get last error details | ||
if lastError := metrics.LastError(); lastError != nil { | ||
fmt.Printf("Last error: %v\n", lastError.Error) | ||
fmt.Printf("Stack trace: %v\n", lastError.Stack) | ||
fmt.Printf("Time: %v\n", lastError.Timestamp) | ||
batch := shrinkmap.BatchOperations[string, int]{ | ||
Operations: []shrinkmap.BatchOperation[string, int]{ | ||
{Type: shrinkmap.BatchSet, Key: "one", Value: 1}, | ||
{Type: shrinkmap.BatchSet, Key: "two", Value: 2}, | ||
{Type: shrinkmap.BatchDelete, Key: "three"}, | ||
}, | ||
} | ||
|
||
// Get error history (last 10 errors) | ||
errorHistory := metrics.ErrorHistory() | ||
for _, err := range errorHistory { | ||
fmt.Printf("Error: %v, Time: %v\n", err.Error, err.Timestamp) | ||
} | ||
// Apply all operations atomically | ||
sm.ApplyBatch(batch) | ||
``` | ||
|
||
### State Inspection | ||
### Safe Iteration | ||
|
||
Safely inspect map state without locking: | ||
Iterate over map contents safely using the iterator: | ||
|
||
```go | ||
// Get current state snapshot | ||
// Create an iterator | ||
iter := sm.NewIterator() | ||
|
||
// Iterate over all items | ||
for iter.Next() { | ||
key, value := iter.Get() | ||
fmt.Printf("Key: %v, Value: %v\n", key, value) | ||
} | ||
|
||
// Or use snapshot for bulk processing | ||
snapshot := sm.Snapshot() | ||
for _, kv := range snapshot { | ||
fmt.Printf("Key: %v, Value: %v\n", kv.Key, kv.Value) | ||
} | ||
``` | ||
|
||
### Resource Management | ||
### Performance Monitoring | ||
|
||
Proper cleanup with graceful shutdown: | ||
Track performance metrics: | ||
|
||
```go | ||
// Create map | ||
sm := shrinkmap.New[string, int](shrinkmap.DefaultConfig()) | ||
metrics := sm.GetMetrics() | ||
fmt.Printf("Total operations: %d\n", metrics.TotalItemsProcessed()) | ||
fmt.Printf("Peak size: %d\n", metrics.PeakSize()) | ||
fmt.Printf("Total shrinks: %d\n", metrics.TotalShrinks()) | ||
``` | ||
|
||
// Ensure cleanup | ||
defer sm.Stop() | ||
## 🔍 Configuration Options | ||
|
||
// Or stop explicitly when needed | ||
sm.Stop() | ||
```go | ||
config := shrinkmap.Config{ | ||
InitialCapacity: 1000, | ||
AutoShrinkEnabled: true, | ||
ShrinkInterval: time.Second, | ||
MinShrinkInterval: time.Second, | ||
ShrinkRatio: 0.5, | ||
CapacityGrowthFactor: 1.5, | ||
MaxMapSize: 1000000, | ||
} | ||
``` | ||
|
||
## Thread Safety Guarantees | ||
## 🛡️ Thread Safety Guarantees | ||
|
||
- All map operations are atomic and thread-safe | ||
- Metrics collection is non-blocking and thread-safe | ||
- Shrinking operations are coordinated to prevent conflicts | ||
- Safe concurrent access from multiple goroutines | ||
- Panic recovery in auto-shrink goroutine | ||
- Thread-safe error tracking and metrics collection | ||
- Safe state inspection with snapshots | ||
- Thread-safe batch operations | ||
- Safe iteration with consistent snapshots | ||
- Coordinated shrinking operations | ||
- Thread-safe metrics collection | ||
|
||
## 📊 Performance | ||
|
||
## Best Practices | ||
Benchmark results on typical operations (Intel i7-9700K, 32GB RAM): | ||
|
||
1. Always ensure proper cleanup: | ||
``` | ||
BenchmarkBasicOperations/Sequential_Set-8 5000000 234 ns/op | ||
BenchmarkBasicOperations/Sequential_Get-8 10000000 112 ns/op | ||
BenchmarkBatchOperations/BatchSize_100-8 100000 15234 ns/op | ||
BenchmarkConcurrency/Parallel_8-8 1000000 1123 ns/op | ||
``` | ||
|
||
## 📝 Best Practices | ||
|
||
1. **Resource Management** | ||
```go | ||
sm := shrinkmap.New[string, int](config) | ||
defer sm.Stop() // Ensure auto-shrink goroutine is cleaned up | ||
defer sm.Stop() // Always ensure proper cleanup | ||
``` | ||
|
||
2. Monitor errors and panics: | ||
2. **Batch Processing** | ||
```go | ||
metrics := sm.GetMetrics() | ||
if metrics.TotalErrors() > 0 { | ||
// Investigate error history | ||
for _, err := range metrics.ErrorHistory() { | ||
log.Printf("Error: %v, Time: %v\n", err.Error, err.Timestamp) | ||
} | ||
} | ||
// Use batch operations for multiple updates | ||
batch := prepareBatchOperations() | ||
sm.ApplyBatch(batch) | ||
``` | ||
|
||
3. Use snapshots for safe iteration: | ||
3. **Safe Iteration** | ||
```go | ||
snapshot := sm.Snapshot() | ||
for _, kv := range snapshot { | ||
// Process items without holding locks | ||
process(kv.Key, kv.Value) | ||
// Use iterator for safe enumeration | ||
iter := sm.NewIterator() | ||
for iter.Next() { | ||
// Process items safely | ||
} | ||
``` | ||
|
||
## Version History | ||
|
||
- 0.0.2 (ing...) | ||
- Added error tracking and panic recovery | ||
- Added state snapshot functionality | ||
- Added graceful shutdown with Stop() | ||
- Enhanced metrics with error statistics | ||
- Improved resource cleanup | ||
- Added comprehensive error tracking tests | ||
|
||
- 0.0.1 | ||
- Initial release | ||
- Thread-safe implementation with atomic operations | ||
- Generic type support | ||
- Automatic shrinking with configurable policies | ||
- Comprehensive benchmark suite | ||
- Race condition free guarantee | ||
## 🗓️ Version History | ||
|
||
### v0.0.3 (Current) | ||
- Added batch operations support for atomic updates | ||
- Implemented safe iterator pattern | ||
- Enhanced performance for bulk operations | ||
- Added comprehensive benchmarking suite | ||
- Improved documentation and examples | ||
|
||
### v0.0.2 | ||
- Added error tracking and panic recovery | ||
- Added state snapshot functionality | ||
- Added graceful shutdown | ||
- Enhanced metrics collection | ||
- Improved resource cleanup | ||
|
||
### v0.0.1 | ||
- Initial release with core functionality | ||
- Thread-safe implementation | ||
- Automatic shrinking support | ||
- Generic type support | ||
|
||
## 📄 License | ||
|
||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. | ||
|
||
--- | ||
Made with ❤️ by [Jongyun Ha](https://github.com/jongyunha) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
package shrinkmap | ||
|
||
// BatchOperations provides batch operation capabilities | ||
type BatchOperations[K comparable, V any] struct { | ||
Operations []BatchOperation[K, V] | ||
} | ||
|
||
type BatchOperation[K comparable, V any] struct { | ||
Type BatchOpType | ||
Key K | ||
Value V | ||
} | ||
|
||
type BatchOpType int | ||
|
||
const ( | ||
BatchSet BatchOpType = iota | ||
BatchDelete | ||
) | ||
|
||
// ApplyBatch applies multiple operations atomically | ||
func (sm *ShrinkableMap[K, V]) ApplyBatch(batch BatchOperations[K, V]) error { | ||
sm.mu.Lock() | ||
defer sm.mu.Unlock() | ||
|
||
for _, op := range batch.Operations { | ||
switch op.Type { | ||
case BatchSet: | ||
_, exists := sm.data[op.Key] | ||
sm.data[op.Key] = op.Value | ||
if !exists { | ||
sm.itemCount.Add(1) | ||
sm.updateMetrics(1) | ||
} | ||
case BatchDelete: | ||
if _, exists := sm.data[op.Key]; exists { | ||
delete(sm.data, op.Key) | ||
sm.deletedCount.Add(1) | ||
} | ||
} | ||
} | ||
|
||
if sm.config.AutoShrinkEnabled { | ||
go sm.TryShrink() | ||
} | ||
return nil | ||
} |
Oops, something went wrong.