13 Commits

Author SHA1 Message Date
Leonel Quinteros
af699a9df0 Add Travis badge 2016-07-15 19:49:54 -03:00
Leonel Quinteros
eaec826ac5 Reduce go versions on travis tests to 1.5 and forward 2016-07-15 19:46:51 -03:00
Leonel Quinteros
653444ba98 Fix race conditions using Anko. 2016-07-15 19:46:17 -03:00
Leonel Quinteros
74daa24696 Plural-Forms formula support. Headers parsing. Multiline strings support. 2016-07-15 19:04:59 -03:00
Leonel Quinteros
af707140e3 Link GNU home from readme 2016-07-12 16:46:55 -03:00
Leonel Quinteros
e1c7cc4c7d Improved readme docs 2016-07-01 12:43:21 -03:00
Leonel Quinteros
4ec3949a9c Improved readme docs 2016-07-01 12:40:59 -03:00
Leonel Quinteros
2981e87657 Set default library location to default path on GNU gettext utilities. 2016-07-01 12:21:59 -03:00
Leonel Quinteros
ad380b8ede Fix Locale.findPO to support language code simplification on LC_MESSAGES dir. 2016-07-01 11:21:13 -03:00
Leonel Quinteros
e62229fc8e Merge pull request #3 from cinghiale/support_LC_MESSAGES
added support for .po files stored in a LC_MESSAGES subdir
2016-07-01 10:59:45 -03:00
David Mugnai
8a0f825cf4 added support for .po files stored in a LC_MESSAGES subdir 2016-07-01 15:41:56 +02:00
Leonel Quinteros
f9c18b1237 Improved docs 2016-06-26 15:52:59 -03:00
Leonel Quinteros
80c24ef4e1 Link v1.0.0 release 2016-06-26 15:46:47 -03:00
8 changed files with 512 additions and 101 deletions

7
.travis.yml Normal file
View File

@@ -0,0 +1,7 @@
language: go
script: go test -v -race ./...
go:
- 1.5
- 1.6
- tip

View File

@@ -1,25 +1,37 @@
[![GoDoc](https://godoc.org/github.com/leonelquinteros/gotext?status.svg)](https://godoc.org/github.com/leonelquinteros/gotext) [![GoDoc](https://godoc.org/github.com/leonelquinteros/gotext?status.svg)](https://godoc.org/github.com/leonelquinteros/gotext)
[![Build Status](https://travis-ci.org/leonelquinteros/gotext.svg?branch=master)](https://travis-ci.org/leonelquinteros/gotext)
# Gotext # Gotext
GNU gettext utilities for Go. [GNU gettext utilities](https://www.gnu.org/software/gettext) for Go.
Version: [0.9.1](https://github.com/leonelquinteros/gotext/releases/tag/v0.9.1) Version: [v1.1.0](https://github.com/leonelquinteros/gotext/releases/tag/v1.1.0)
#Features # Features
- Implements GNU gettext support in native Go. - Implements GNU gettext support in native Go.
- Safe for concurrent use across multiple goroutines. - Complete support for [PO files](https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html) including:
- Support for multiline strings and headers.
- Support for variables inside translation strings using Go's [fmt syntax](https://golang.org/pkg/fmt/).
- Support for [pluralization rules](https://www.gnu.org/software/gettext/manual/html_node/Translating-plural-forms.html).
- Support for [message contexts](https://www.gnu.org/software/gettext/manual/html_node/Contexts.html).
- Thread-safe: This package is safe for concurrent use across multiple goroutines.
- It works with UTF-8 encoding as it's the default for Go language. - It works with UTF-8 encoding as it's the default for Go language.
- Unit tests available. - Unit tests available.
- Language codes are automatically simplified from the form "en_UK" to "en" if the first isn't available. - Language codes are automatically simplified from the form `en_UK` to `en` if the first isn't available.
- Ready to use inside Go templates. - Ready to use inside Go templates.
- Support for [pluralization rules](https://www.gnu.org/software/gettext/manual/html_node/Plural-forms.html).
- Support for [message context](https://www.gnu.org/software/gettext/manual/html_node/Contexts.html).
- Support for variables inside translation strings using Go's [fmt package syntax](https://golang.org/pkg/fmt/).
# License
[MIT license](LICENSE)
# Documentation
Refer to the Godoc package documentation at (https://godoc.org/github.com/leonelquinteros/gotext)
# Installation # Installation
@@ -32,14 +44,30 @@ go get github.com/leonelquinteros/gotext
- No need for environment variables. Some naming conventions are applied but not needed. - No need for environment variables. Some naming conventions are applied but not needed.
# License #### Version vendoring
[MIT license](LICENSE) Stable releases use [semantic versioning](http://semver.org/spec/v2.0.0.html) tagging on this repository.
You can rely on this to use your preferred vendoring tool or to manually retrieve the corresponding release tag from the GitHub repository.
# Documentation ##### Vendoring with [gopkg.in](http://labix.org/gopkg.in)
Refer to the Godoc package documentation at (https://godoc.org/github.com/leonelquinteros/gotext) [http://gopkg.in/leonelquinteros/gotext.v1](http://gopkg.in/leonelquinteros/gotext.v1)
To get the latest v1 package stable release, execute:
```
go get gopkg.in/leonelquinteros/gotext.v1
```
To import this package, add the following line to your code:
```go
import "gopkg.in/leonelquinteros/gotext.v1"
```
Refer to it as gotext.
# Locales directories structure # Locales directories structure
@@ -49,25 +77,29 @@ or to object constructors depending on the use, but either will use the same con
Inside the base directory where will be the language directories named using the language and country 2-letter codes (en_US, es_AR, ...). Inside the base directory where will be the language directories named using the language and country 2-letter codes (en_US, es_AR, ...).
All package functions can lookup after the simplified version for each language in case the full code isn't present but the more general language code exists. All package functions can lookup after the simplified version for each language in case the full code isn't present but the more general language code exists.
So if the language set is "en_UK", but there is no directory named after that code and there is a directory named "en", So if the language set is `en_UK`, but there is no directory named after that code and there is a directory named `en`,
all package functions will be able to resolve this generalization and provide translations for the more general library. all package functions will be able to resolve this generalization and provide translations for the more general library.
The language codes are assumed to be [ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) codes (2-letter codes). The language codes are assumed to be [ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) codes (2-letter codes).
That said, most functions will work with any coding standard as long the directory name matches the language code set on the configuration. That said, most functions will work with any coding standard as long the directory name matches the language code set on the configuration.
A normal library directory structure may look like: Then, there can be a `LC_MESSAGES` containing all PO files or the PO files themselves.
A library directory structure can look like:
``` ```
/path/to/locales /path/to/locales
/path/to/locales/en_US /path/to/locales/en_US
/path/to/locales/en_US/default.po /path/to/locales/en_US/LC_MESSAGES
/path/to/locales/en_US/extras.po /path/to/locales/en_US/LC_MESSAGES/default.po
/path/to/locales/en_US/LC_MESSAGES/extras.po
/path/to/locales/en_UK /path/to/locales/en_UK
/path/to/locales/en_UK/default.po /path/to/locales/en_UK/LC_MESSAGES
/path/to/locales/en_UK/extras.po /path/to/locales/en_UK/LC_MESSAGES/default.po
/path/to/locales/en_UK/LC_MESSAGES/extras.po
/path/to/locales/en_AU /path/to/locales/en_AU
/path/to/locales/en_AU/default.po /path/to/locales/en_AU/LC_MESSAGES
/path/to/locales/en_AU/extras.po /path/to/locales/en_AU/LC_MESSAGES/default.po
/path/to/locales/en_AU/LC_MESSAGES/extras.po
/path/to/locales/es /path/to/locales/es
/path/to/locales/es/default.po /path/to/locales/es/default.po
/path/to/locales/es/extras.po /path/to/locales/es/extras.po
@@ -85,9 +117,9 @@ And so on...
# About translation function names # About translation function names
The standard GNU gettext defines helper functions that maps to the gettext() function and it's widely adopted by most implementations. The standard GNU gettext defines helper functions that maps to the `gettext()` function and it's widely adopted by most implementations.
The basic translation function is usually _() in the form: The basic translation function is usually `_()` in the form:
``` ```
_("Translate this") _("Translate this")
@@ -110,7 +142,7 @@ func _(str string, vars ...interface{}) string {
This is valid and can be used within a package. This is valid and can be used within a package.
In normal conditions the Go compiler will optimize the calls to _() by replacing its content in place of the function call to reduce the function calling overhead. In normal conditions the Go compiler will optimize the calls to `_()` by replacing its content in place of the function call to reduce the function calling overhead.
This is a normal Go compiler behavior. This is a normal Go compiler behavior.
@@ -226,6 +258,10 @@ msgstr "This one sets the var: %s"
## Use plural forms of translations ## Use plural forms of translations
PO format supports defining one or more plural forms for the same translation. PO format supports defining one or more plural forms for the same translation.
Relying on the PO file headers, a Plural-Forms formula can be set on the translation file
as defined in (https://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/Plural-forms.html)
Plural formulas are parsed and evaluated using [Anko](https://github.com/mattn/anko)
```go ```go
import "github.com/leonelquinteros/gotext" import "github.com/leonelquinteros/gotext"
@@ -233,6 +269,12 @@ import "github.com/leonelquinteros/gotext"
func main() { func main() {
// Set PO content // Set PO content
str := ` str := `
msgid ""
msgstr ""
# Header below
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
msgid "Translate this" msgid "Translate this"
msgstr "Translated text" msgstr "Translated text"
@@ -243,15 +285,14 @@ msgid "One with var: %s"
msgid_plural "Several with vars: %s" msgid_plural "Several with vars: %s"
msgstr[0] "This one is the singular: %s" msgstr[0] "This one is the singular: %s"
msgstr[1] "This one is the plural: %s" msgstr[1] "This one is the plural: %s"
msgstr[2] "And this is the second plural form: %s"
` `
// Create Po object // Create Po object
po := new(Po) po := new(Po)
po.Parse(str) po.Parse(str)
println(po.GetN("One with var: %s", "Several with vars: %s", 2, v)) println(po.GetN("One with var: %s", "Several with vars: %s", 54, v))
// "And this is the second plural form: Variable" // "This one is the plural: Variable"
} }
``` ```

View File

@@ -1,8 +1,6 @@
/* /*
Package gotext implements GNU gettext utilities. Package gotext implements GNU gettext utilities.
Version 0.9.0 (stable)
For quick/simple translations you can use the package level functions directly. For quick/simple translations you can use the package level functions directly.
import "github.com/leonelquinteros/gotext" import "github.com/leonelquinteros/gotext"
@@ -30,7 +28,7 @@ var (
language = "en_US" language = "en_US"
// Path to library directory where all locale directories and translation files are. // Path to library directory where all locale directories and translation files are.
library = "/tmp" library = "/usr/local/share/locale"
// Storage for package level methods // Storage for package level methods
storage *Locale storage *Locale
@@ -102,8 +100,7 @@ func Get(str string, vars ...interface{}) string {
return GetD(domain, str, vars...) return GetD(domain, str, vars...)
} }
// GetN retrieves the (N)th plural form translation for the given string in the "default" domain. // GetN retrieves the (N)th plural form of translation for the given string in the "default" domain.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func GetN(str, plural string, n int, vars ...interface{}) string { func GetN(str, plural string, n int, vars ...interface{}) string {
return GetND("default", str, plural, n, vars...) return GetND("default", str, plural, n, vars...)
@@ -112,10 +109,10 @@ func GetN(str, plural string, n int, vars ...interface{}) string {
// GetD returns the corresponding translation in the given domain for a given string. // GetD returns the corresponding translation in the given domain for a given string.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func GetD(dom, str string, vars ...interface{}) string { func GetD(dom, str string, vars ...interface{}) string {
return GetND(dom, str, str, 0, vars...) return GetND(dom, str, str, 1, vars...)
} }
// GetND retrieves the (N)th plural form translation in the given domain for a given string. // GetND retrieves the (N)th plural form of translation in the given domain for a given string.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func GetND(dom, str, plural string, n int, vars ...interface{}) string { func GetND(dom, str, plural string, n int, vars ...interface{}) string {
// Try to load default package Locale storage // Try to load default package Locale storage
@@ -131,8 +128,7 @@ func GetC(str, ctx string, vars ...interface{}) string {
return GetDC(domain, str, ctx, vars...) return GetDC(domain, str, ctx, vars...)
} }
// GetNC retrieves the (N)th plural form translation for the given string in the given context in the "default" domain. // GetNC retrieves the (N)th plural form of translation for the given string in the given context in the "default" domain.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func GetNC(str, plural string, n int, ctx string, vars ...interface{}) string { func GetNC(str, plural string, n int, ctx string, vars ...interface{}) string {
return GetNDC("default", str, plural, n, ctx, vars...) return GetNDC("default", str, plural, n, ctx, vars...)
@@ -141,10 +137,10 @@ func GetNC(str, plural string, n int, ctx string, vars ...interface{}) string {
// GetDC returns the corresponding translation in the given domain for the given string in the given context. // GetDC returns the corresponding translation in the given domain for the given string in the given context.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func GetDC(dom, str, ctx string, vars ...interface{}) string { func GetDC(dom, str, ctx string, vars ...interface{}) string {
return GetNDC(dom, str, str, 0, ctx, vars...) return GetNDC(dom, str, str, 1, ctx, vars...)
} }
// GetNDC retrieves the (N)th plural form translation in the given domain for a given string. // GetNDC retrieves the (N)th plural form of translation in the given domain for a given string.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func GetNDC(dom, str, plural string, n int, ctx string, vars ...interface{}) string { func GetNDC(dom, str, plural string, n int, ctx string, vars ...interface{}) string {
// Try to load default package Locale storage // Try to load default package Locale storage

View File

@@ -31,7 +31,17 @@ func TestGettersSetters(t *testing.T) {
func TestPackageFunctions(t *testing.T) { func TestPackageFunctions(t *testing.T) {
// Set PO content // Set PO content
str := `# Some comment str := `
msgid ""
msgstr ""
# Initial comment
# Headers below
"Language: en\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
# Some comment
msgid "My text" msgid "My text"
msgstr "Translated text" msgstr "Translated text"
@@ -109,8 +119,8 @@ msgstr "More translation"
// Test plural // Test plural
tr = GetN("One with var: %s", "Several with vars: %s", 2, v) tr = GetN("One with var: %s", "Several with vars: %s", 2, v)
if tr != "And this is the second plural form: Variable" { if tr != "This one is the plural: Variable" {
t.Errorf("Expected 'And this is the second plural form: Variable' but got '%s'", tr) t.Errorf("Expected 'This one is the plural: Variable' but got '%s'", tr)
} }
// Test context translations // Test context translations
@@ -125,7 +135,7 @@ msgstr "More translation"
t.Errorf("Expected 'This one is the singular in a Ctx context: Variable' but got '%s'", tr) t.Errorf("Expected 'This one is the singular in a Ctx context: Variable' but got '%s'", tr)
} }
tr = GetNC("One with var: %s", "Several with vars: %s", 1, "Ctx", v) tr = GetNC("One with var: %s", "Several with vars: %s", 19, "Ctx", v)
if tr != "This one is the plural in a Ctx context: Variable" { if tr != "This one is the plural in a Ctx context: Variable" {
t.Errorf("Expected 'This one is the plural in a Ctx context: Variable' but got '%s'", tr) t.Errorf("Expected 'This one is the plural in a Ctx context: Variable' but got '%s'", tr)
} }

View File

@@ -20,13 +20,13 @@ Example:
// Create Locale with library path and language code // Create Locale with library path and language code
l := gotext.NewLocale("/path/to/i18n/dir", "en_US") l := gotext.NewLocale("/path/to/i18n/dir", "en_US")
// Load domain '/path/to/i18n/dir/en_US/default.po' // Load domain '/path/to/i18n/dir/en_US/LC_MESSAGES/default.po'
l.AddDomain("default") l.AddDomain("default")
// Translate text from default domain // Translate text from default domain
println(l.Get("Translate this")) println(l.Get("Translate this"))
// Load different domain ('/path/to/i18n/dir/en_US/extras.po') // Load different domain ('/path/to/i18n/dir/en_US/LC_MESSAGES/extras.po')
l.AddDomain("extras") l.AddDomain("extras")
// Translate text from domain // Translate text from domain
@@ -58,23 +58,38 @@ func NewLocale(p, l string) *Locale {
} }
} }
func (l *Locale) findPO(dom string) string {
filename := path.Join(l.path, l.lang, "LC_MESSAGES", dom+".po")
if _, err := os.Stat(filename); err == nil {
return filename
}
if len(l.lang) > 2 {
filename = path.Join(l.path, l.lang[:2], "LC_MESSAGES", dom+".po")
if _, err := os.Stat(filename); err == nil {
return filename
}
}
filename = path.Join(l.path, l.lang, dom+".po")
if _, err := os.Stat(filename); err == nil {
return filename
}
if len(l.lang) > 2 {
filename = path.Join(l.path, l.lang[:2], dom+".po")
}
return filename
}
// AddDomain creates a new domain for a given locale object and initializes the Po object. // AddDomain creates a new domain for a given locale object and initializes the Po object.
// If the domain exists, it gets reloaded. // If the domain exists, it gets reloaded.
func (l *Locale) AddDomain(dom string) { func (l *Locale) AddDomain(dom string) {
po := new(Po) po := new(Po)
// Check for file.
filename := path.Clean(l.path + string(os.PathSeparator) + l.lang + string(os.PathSeparator) + dom + ".po")
// Try to use the generic language dir if the provided isn't available
if _, err := os.Stat(filename); err != nil {
if len(l.lang) > 2 {
filename = path.Clean(l.path + string(os.PathSeparator) + l.lang[:2] + string(os.PathSeparator) + dom + ".po")
}
}
// Parse file. // Parse file.
po.ParseFile(filename) po.ParseFile(l.findPO(dom))
// Save new domain // Save new domain
l.Lock() l.Lock()
@@ -92,8 +107,7 @@ func (l *Locale) Get(str string, vars ...interface{}) string {
return l.GetD("default", str, vars...) return l.GetD("default", str, vars...)
} }
// GetN retrieves the (N)th plural form translation for the given string in the "default" domain. // GetN retrieves the (N)th plural form of translation for the given string in the "default" domain.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (l *Locale) GetN(str, plural string, n int, vars ...interface{}) string { func (l *Locale) GetN(str, plural string, n int, vars ...interface{}) string {
return l.GetND("default", str, plural, n, vars...) return l.GetND("default", str, plural, n, vars...)
@@ -102,11 +116,10 @@ func (l *Locale) GetN(str, plural string, n int, vars ...interface{}) string {
// GetD returns the corresponding translation in the given domain for the given string. // GetD returns the corresponding translation in the given domain for the given string.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (l *Locale) GetD(dom, str string, vars ...interface{}) string { func (l *Locale) GetD(dom, str string, vars ...interface{}) string {
return l.GetND(dom, str, str, 0, vars...) return l.GetND(dom, str, str, 1, vars...)
} }
// GetND retrieves the (N)th plural form translation in the given domain for the given string. // GetND retrieves the (N)th plural form of translation in the given domain for the given string.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (l *Locale) GetND(dom, str, plural string, n int, vars ...interface{}) string { func (l *Locale) GetND(dom, str, plural string, n int, vars ...interface{}) string {
// Sync read // Sync read
@@ -131,8 +144,7 @@ func (l *Locale) GetC(str, ctx string, vars ...interface{}) string {
return l.GetDC("default", str, ctx, vars...) return l.GetDC("default", str, ctx, vars...)
} }
// GetNC retrieves the (N)th plural form translation for the given string in the given context in the "default" domain. // GetNC retrieves the (N)th plural form of translation for the given string in the given context in the "default" domain.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (l *Locale) GetNC(str, plural string, n int, ctx string, vars ...interface{}) string { func (l *Locale) GetNC(str, plural string, n int, ctx string, vars ...interface{}) string {
return l.GetNDC("default", str, plural, n, ctx, vars...) return l.GetNDC("default", str, plural, n, ctx, vars...)
@@ -141,11 +153,10 @@ func (l *Locale) GetNC(str, plural string, n int, ctx string, vars ...interface{
// GetDC returns the corresponding translation in the given domain for the given string in the given context. // GetDC returns the corresponding translation in the given domain for the given string in the given context.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (l *Locale) GetDC(dom, str, ctx string, vars ...interface{}) string { func (l *Locale) GetDC(dom, str, ctx string, vars ...interface{}) string {
return l.GetNDC(dom, str, str, 0, ctx, vars...) return l.GetNDC(dom, str, str, 1, ctx, vars...)
} }
// GetNDC retrieves the (N)th plural form translation in the given domain for the given string in the given context. // GetNDC retrieves the (N)th plural form of translation in the given domain for the given string in the given context.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (l *Locale) GetNDC(dom, str, plural string, n int, ctx string, vars ...interface{}) string { func (l *Locale) GetNDC(dom, str, plural string, n int, ctx string, vars ...interface{}) string {
// Sync read // Sync read

View File

@@ -8,7 +8,17 @@ import (
func TestLocale(t *testing.T) { func TestLocale(t *testing.T) {
// Set PO content // Set PO content
str := `# Some comment str := `
msgid ""
msgstr ""
# Initial comment
# Headers below
"Language: en\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
# Some comment
msgid "My text" msgid "My text"
msgstr "Translated text" msgstr "Translated text"
@@ -49,14 +59,14 @@ msgstr "More translation"
` `
// Create Locales directory with simplified language code // Create Locales directory with simplified language code
dirname := path.Clean("/tmp" + string(os.PathSeparator) + "en") dirname := path.Join("/tmp", "en", "LC_MESSAGES")
err := os.MkdirAll(dirname, os.ModePerm) err := os.MkdirAll(dirname, os.ModePerm)
if err != nil { if err != nil {
t.Fatalf("Can't create test directory: %s", err.Error()) t.Fatalf("Can't create test directory: %s", err.Error())
} }
// Write PO content to file // Write PO content to file
filename := path.Clean(dirname + string(os.PathSeparator) + "my_domain.po") filename := path.Join(dirname, "my_domain.po")
f, err := os.Create(filename) f, err := os.Create(filename)
if err != nil { if err != nil {
@@ -91,9 +101,9 @@ msgstr "More translation"
} }
// Test plural // Test plural
tr = l.GetND("my_domain", "One with var: %s", "Several with vars: %s", 2, v) tr = l.GetND("my_domain", "One with var: %s", "Several with vars: %s", 7, v)
if tr != "And this is the second plural form: Variable" { if tr != "This one is the plural: Variable" {
t.Errorf("Expected 'And this is the second plural form: Variable' but got '%s'", tr) t.Errorf("Expected 'This one is the plural: Variable' but got '%s'", tr)
} }
// Test non-existent "deafult" domain responses // Test non-existent "deafult" domain responses
@@ -137,7 +147,7 @@ msgstr "More translation"
} }
// Test plural // Test plural
tr = l.GetNDC("my_domain", "One with var: %s", "Several with vars: %s", 1, "Ctx", v) tr = l.GetNDC("my_domain", "One with var: %s", "Several with vars: %s", 3, "Ctx", v)
if tr != "This one is the plural in a Ctx context: Test" { if tr != "This one is the plural in a Ctx context: Test" {
t.Errorf("Expected 'This one is the plural in a Ctx context: Test' but got '%s'", tr) t.Errorf("Expected 'This one is the plural in a Ctx context: Test' but got '%s'", tr)
} }
@@ -168,14 +178,14 @@ msgstr[2] "And this is the second plural form: %s"
` `
// Create Locales directory with simplified language code // Create Locales directory with simplified language code
dirname := path.Clean("/tmp" + string(os.PathSeparator) + "es") dirname := path.Join("/tmp", "es")
err := os.MkdirAll(dirname, os.ModePerm) err := os.MkdirAll(dirname, os.ModePerm)
if err != nil { if err != nil {
t.Fatalf("Can't create test directory: %s", err.Error()) t.Fatalf("Can't create test directory: %s", err.Error())
} }
// Write PO content to file // Write PO content to file
filename := path.Clean(dirname + string(os.PathSeparator) + "race.po") filename := path.Join(dirname, "race.po")
f, err := os.Create(filename) f, err := os.Create(filename)
if err != nil { if err != nil {

153
po.go
View File

@@ -1,8 +1,11 @@
package gotext package gotext
import ( import (
"bufio"
"fmt" "fmt"
"github.com/leonelquinteros/anko/vm"
"io/ioutil" "io/ioutil"
"net/textproto"
"os" "os"
"strconv" "strconv"
"strings" "strings"
@@ -44,8 +47,8 @@ func (t *translation) getN(n int) string {
/* /*
Po parses the content of any PO file and provides all the translation functions needed. Po parses the content of any PO file and provides all the translation functions needed.
It's the base object used by all packafe methods. It's the base object used by all package methods.
And it's safe for concurrent use by multiple goroutines by using the sync package for write locking. And it's safe for concurrent use by multiple goroutines by using the sync package for locking.
Example: Example:
@@ -64,6 +67,19 @@ Example:
*/ */
type Po struct { type Po struct {
// Headers
RawHeaders string
// Language header
Language string
// Plural-Forms header
PluralForms string
// Parsed Plural-Forms header values
nplurals int
plural string
// Storage // Storage
translations map[string]*translation translations map[string]*translation
contexts map[string]map[string]*translation contexts map[string]map[string]*translation
@@ -96,12 +112,14 @@ func (po *Po) ParseFile(f string) {
// Parse loads the translations specified in the provided string (str) // Parse loads the translations specified in the provided string (str)
func (po *Po) Parse(str string) { func (po *Po) Parse(str string) {
// Lock while parsing
po.Lock()
defer po.Unlock()
// Init storage // Init storage
if po.translations == nil { if po.translations == nil {
po.Lock()
po.translations = make(map[string]*translation) po.translations = make(map[string]*translation)
po.contexts = make(map[string]map[string]*translation) po.contexts = make(map[string]map[string]*translation)
po.Unlock()
} }
// Get lines // Get lines
@@ -123,14 +141,13 @@ func (po *Po) Parse(str string) {
} }
// Skip invalid lines // Skip invalid lines
if !strings.HasPrefix(l, "msgctxt") && !strings.HasPrefix(l, "msgid") && !strings.HasPrefix(l, "msgid_plural") && !strings.HasPrefix(l, "msgstr") { if !strings.HasPrefix(l, "\"") && !strings.HasPrefix(l, "msgctxt") && !strings.HasPrefix(l, "msgid") && !strings.HasPrefix(l, "msgid_plural") && !strings.HasPrefix(l, "msgstr") {
continue continue
} }
// Buffer context and continue // Buffer context and continue
if strings.HasPrefix(l, "msgctxt") { if strings.HasPrefix(l, "msgctxt") {
// Save current translation buffer. // Save current translation buffer.
po.Lock()
// No context // No context
if ctx == "" { if ctx == "" {
po.translations[tr.id] = tr po.translations[tr.id] = tr
@@ -141,7 +158,6 @@ func (po *Po) Parse(str string) {
} }
po.contexts[ctx][tr.id] = tr po.contexts[ctx][tr.id] = tr
} }
po.Unlock()
// Flush buffer // Flush buffer
tr = newTranslation() tr = newTranslation()
@@ -158,9 +174,7 @@ func (po *Po) Parse(str string) {
if strings.HasPrefix(l, "msgid") && !strings.HasPrefix(l, "msgid_plural") { if strings.HasPrefix(l, "msgid") && !strings.HasPrefix(l, "msgid_plural") {
// Save current translation buffer if not inside a context. // Save current translation buffer if not inside a context.
if ctx == "" { if ctx == "" {
po.Lock()
po.translations[tr.id] = tr po.translations[tr.id] = tr
po.Unlock()
// Flush buffer // Flush buffer
tr = newTranslation() tr = newTranslation()
@@ -198,21 +212,21 @@ func (po *Po) Parse(str string) {
// Check for indexed translation forms // Check for indexed translation forms
if strings.HasPrefix(l, "[") { if strings.HasPrefix(l, "[") {
in := strings.Index(l, "]") idx := strings.Index(l, "]")
if in == -1 { if idx == -1 {
// Skip wrong index formatting // Skip wrong index formatting
continue continue
} }
// Parse index // Parse index
i, err := strconv.Atoi(l[1:in]) i, err := strconv.Atoi(l[1:idx])
if err != nil { if err != nil {
// Skip wrong index formatting // Skip wrong index formatting
continue continue
} }
// Parse translation string // Parse translation string
tr.trs[i], _ = strconv.Unquote(strings.TrimSpace(l[in+1:])) tr.trs[i], _ = strconv.Unquote(strings.TrimSpace(l[idx+1:]))
// Loop // Loop
continue continue
@@ -220,12 +234,36 @@ func (po *Po) Parse(str string) {
// Save single translation form under 0 index // Save single translation form under 0 index
tr.trs[0], _ = strconv.Unquote(l) tr.trs[0], _ = strconv.Unquote(l)
// Loop
continue
}
// Multi line strings and headers
if strings.HasPrefix(l, "\"") && strings.HasSuffix(l, "\"") {
// Check for multiline from previously set msgid
if tr.id != "" {
// Append to last translation found
uq, _ := strconv.Unquote(l)
tr.trs[len(tr.trs)-1] += uq
// Loop
continue
}
// Otherwise is a header
h, err := strconv.Unquote(strings.TrimSpace(l))
if err != nil {
continue
}
po.RawHeaders += h
continue
} }
} }
// Save last translation buffer. // Save last translation buffer.
if tr.id != "" { if tr.id != "" {
po.Lock()
if ctx == "" { if ctx == "" {
po.translations[tr.id] = tr po.translations[tr.id] = tr
} else { } else {
@@ -235,8 +273,83 @@ func (po *Po) Parse(str string) {
} }
po.contexts[ctx][tr.id] = tr po.contexts[ctx][tr.id] = tr
} }
po.Unlock()
} }
// Parse headers
po.RawHeaders += "\n\n"
reader := bufio.NewReader(strings.NewReader(po.RawHeaders))
tp := textproto.NewReader(reader)
mimeHeader, err := tp.ReadMIMEHeader()
if err != nil {
return
}
// Get/save needed headers
po.Language = mimeHeader.Get("Language")
po.PluralForms = mimeHeader.Get("Plural-Forms")
// Parse Plural-Forms formula
if po.PluralForms == "" {
return
}
// Split plural form header value
pfs := strings.Split(po.PluralForms, ";")
// Parse values
for _, i := range pfs {
vs := strings.SplitN(i, "=", 2)
if len(vs) != 2 {
continue
}
switch strings.TrimSpace(vs[0]) {
case "nplurals":
po.nplurals, _ = strconv.Atoi(vs[1])
case "plural":
po.plural = vs[1]
}
}
}
// pluralForm calculates the plural form index corresponding to n.
// Returns 0 on error
func (po *Po) pluralForm(n int) int {
po.RLock()
defer po.RUnlock()
// Failsafe
if po.nplurals < 1 {
return 0
}
if po.plural == "" {
return 0
}
// Init compiler
env := vm.NewEnv()
env.Define("n", n)
plural, err := env.Execute(po.plural)
if err != nil {
return 0
}
if plural.Type().Name() == "bool" {
if plural.Bool() {
return 1
} else {
return 0
}
}
if int(plural.Int()) > po.nplurals {
return 0
}
return int(plural.Int())
} }
// Get retrieves the corresponding translation for the given string. // Get retrieves the corresponding translation for the given string.
@@ -256,8 +369,7 @@ func (po *Po) Get(str string, vars ...interface{}) string {
return fmt.Sprintf(str, vars...) return fmt.Sprintf(str, vars...)
} }
// GetN retrieves the (N)th plural form translation for the given string. // GetN retrieves the (N)th plural form of translation for the given string.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (po *Po) GetN(str, plural string, n int, vars ...interface{}) string { func (po *Po) GetN(str, plural string, n int, vars ...interface{}) string {
// Sync read // Sync read
@@ -266,7 +378,7 @@ func (po *Po) GetN(str, plural string, n int, vars ...interface{}) string {
if po.translations != nil { if po.translations != nil {
if _, ok := po.translations[str]; ok { if _, ok := po.translations[str]; ok {
return fmt.Sprintf(po.translations[str].getN(n), vars...) return fmt.Sprintf(po.translations[str].getN(po.pluralForm(n)), vars...)
} }
} }
@@ -295,8 +407,7 @@ func (po *Po) GetC(str, ctx string, vars ...interface{}) string {
return fmt.Sprintf(str, vars...) return fmt.Sprintf(str, vars...)
} }
// GetNC retrieves the (N)th plural form translation for the given string in the given context. // GetNC retrieves the (N)th plural form of translation for the given string in the given context.
// If n == 0, usually the singular form of the string is returned as defined in the PO file.
// Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax. // Supports optional parameters (vars... interface{}) to be inserted on the formatted string using the fmt.Printf syntax.
func (po *Po) GetNC(str, plural string, n int, ctx string, vars ...interface{}) string { func (po *Po) GetNC(str, plural string, n int, ctx string, vars ...interface{}) string {
// Sync read // Sync read
@@ -307,7 +418,7 @@ func (po *Po) GetNC(str, plural string, n int, ctx string, vars ...interface{})
if _, ok := po.contexts[ctx]; ok { if _, ok := po.contexts[ctx]; ok {
if po.contexts[ctx] != nil { if po.contexts[ctx] != nil {
if _, ok := po.contexts[ctx][str]; ok { if _, ok := po.contexts[ctx][str]; ok {
return fmt.Sprintf(po.contexts[ctx][str].getN(n), vars...) return fmt.Sprintf(po.contexts[ctx][str].getN(po.pluralForm(n)), vars...)
} }
} }
} }

View File

@@ -8,7 +8,17 @@ import (
func TestPo(t *testing.T) { func TestPo(t *testing.T) {
// Set PO content // Set PO content
str := `# Some comment str := `
msgid ""
msgstr ""
# Initial comment
# Headers below
"Language: en\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
# Some comment
msgid "My text" msgid "My text"
msgstr "Translated text" msgstr "Translated text"
@@ -16,6 +26,11 @@ msgstr "Translated text"
msgid "Another string" msgid "Another string"
msgstr "" msgstr ""
#Multi-line string
msgid "Multi-line"
msgstr "Multi "
"line"
msgid "One with var: %s" msgid "One with var: %s"
msgid_plural "Several with vars: %s" msgid_plural "Several with vars: %s"
msgstr[0] "This one is the singular: %s" msgstr[0] "This one is the singular: %s"
@@ -82,10 +97,16 @@ msgstr "More translation"
t.Errorf("Expected 'This one is the singular: Variable' but got '%s'", tr) t.Errorf("Expected 'This one is the singular: Variable' but got '%s'", tr)
} }
// Test multi-line
tr = po.Get("Multi-line")
if tr != "Multi line" {
t.Errorf("Expected 'Multi line' but got '%s'", tr)
}
// Test plural // Test plural
tr = po.GetN("One with var: %s", "Several with vars: %s", 2, v) tr = po.GetN("One with var: %s", "Several with vars: %s", 2, v)
if tr != "And this is the second plural form: Variable" { if tr != "This one is the plural: Variable" {
t.Errorf("Expected 'And this is the second plural form: Variable' but got '%s'", tr) t.Errorf("Expected 'This one is the plural: Variable' but got '%s'", tr)
} }
// Test inexistent translations // Test inexistent translations
@@ -94,7 +115,7 @@ msgstr "More translation"
t.Errorf("Expected 'This is a test' but got '%s'", tr) t.Errorf("Expected 'This is a test' but got '%s'", tr)
} }
tr = po.GetN("This is a test", "This are tests", 1) tr = po.GetN("This is a test", "This are tests", 100)
if tr != "This are tests" { if tr != "This are tests" {
t.Errorf("Expected 'This are tests' but got '%s'", tr) t.Errorf("Expected 'This are tests' but got '%s'", tr)
} }
@@ -105,7 +126,7 @@ msgstr "More translation"
t.Errorf("Expected '' but got '%s'", tr) t.Errorf("Expected '' but got '%s'", tr)
} }
tr = po.GetN("This one has invalid syntax translations", "This are tests", 1) tr = po.GetN("This one has invalid syntax translations", "This are tests", 4)
if tr != "Plural index" { if tr != "Plural index" {
t.Errorf("Expected 'Plural index' but got '%s'", tr) t.Errorf("Expected 'Plural index' but got '%s'", tr)
} }
@@ -118,7 +139,7 @@ msgstr "More translation"
} }
// Test plural // Test plural
tr = po.GetNC("One with var: %s", "Several with vars: %s", 1, "Ctx", v) tr = po.GetNC("One with var: %s", "Several with vars: %s", 17, "Ctx", v)
if tr != "This one is the plural in a Ctx context: Test" { if tr != "This one is the plural in a Ctx context: Test" {
t.Errorf("Expected 'This one is the plural in a Ctx context: Test' but got '%s'", tr) t.Errorf("Expected 'This one is the plural in a Ctx context: Test' but got '%s'", tr)
} }
@@ -131,6 +152,210 @@ msgstr "More translation"
} }
func TestPoHeaders(t *testing.T) {
// Set PO content
str := `
msgid ""
msgstr ""
# Initial comment
# Headers below
"Language: en\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
# Some comment
msgid "Example"
msgstr "Translated example"
`
// Create po object
po := new(Po)
// Parse
po.Parse(str)
// Check headers expected
if po.Language != "en" {
t.Errorf("Expected 'Language: en' but got '%s'", po.Language)
}
// Check headers expected
if po.PluralForms != "nplurals=2; plural=(n != 1);" {
t.Errorf("Expected 'Plural-Forms: nplurals=2; plural=(n != 1);' but got '%s'", po.PluralForms)
}
}
func TestPluralForms(t *testing.T) {
// Single form
str := `
"Plural-Forms: nplurals=1; plural=0;"
# Some comment
msgid "Singular"
msgid_plural "Plural"
msgstr[0] "Singular form"
msgstr[1] "Plural form 1"
msgstr[2] "Plural form 2"
msgstr[3] "Plural form 3"
`
// Create po object
po := new(Po)
// Parse
po.Parse(str)
// Check plural form
n := po.pluralForm(0)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(0), got %d", n)
}
n = po.pluralForm(1)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(1), got %d", n)
}
n = po.pluralForm(2)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(2), got %d", n)
}
n = po.pluralForm(3)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(3), got %d", n)
}
n = po.pluralForm(50)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(50), got %d", n)
}
// ------------------------------------------------------------------------
// 2 forms
str = `
"Plural-Forms: nplurals=2; plural=n != 1;"
# Some comment
msgid "Singular"
msgid_plural "Plural"
msgstr[0] "Singular form"
msgstr[1] "Plural form 1"
msgstr[2] "Plural form 2"
msgstr[3] "Plural form 3"
`
// Create po object
po = new(Po)
// Parse
po.Parse(str)
// Check plural form
n = po.pluralForm(0)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(0), got %d", n)
}
n = po.pluralForm(1)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(1), got %d", n)
}
n = po.pluralForm(2)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(2), got %d", n)
}
n = po.pluralForm(3)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(3), got %d", n)
}
// ------------------------------------------------------------------------
// 3 forms
str = `
"Plural-Forms: nplurals=3; plural=n%10==1 && n%100!=11 ? 0 : n != 0 ? 1 : 2;"
# Some comment
msgid "Singular"
msgid_plural "Plural"
msgstr[0] "Singular form"
msgstr[1] "Plural form 1"
msgstr[2] "Plural form 2"
msgstr[3] "Plural form 3"
`
// Create po object
po = new(Po)
// Parse
po.Parse(str)
// Check plural form
n = po.pluralForm(0)
if n != 2 {
t.Errorf("Expected 2 for pluralForm(0), got %d", n)
}
n = po.pluralForm(1)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(1), got %d", n)
}
n = po.pluralForm(2)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(2), got %d", n)
}
n = po.pluralForm(3)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(3), got %d", n)
}
n = po.pluralForm(100)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(100), got %d", n)
}
n = po.pluralForm(49)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(3), got %d", n)
}
// ------------------------------------------------------------------------
// 3 forms special
str = `
"Plural-Forms: nplurals=3;"
"plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2;"
# Some comment
msgid "Singular"
msgid_plural "Plural"
msgstr[0] "Singular form"
msgstr[1] "Plural form 1"
msgstr[2] "Plural form 2"
msgstr[3] "Plural form 3"
`
// Create po object
po = new(Po)
// Parse
po.Parse(str)
// Check plural form
n = po.pluralForm(1)
if n != 0 {
t.Errorf("Expected 0 for pluralForm(1), got %d", n)
}
n = po.pluralForm(2)
if n != 1 {
t.Errorf("Expected 1 for pluralForm(2), got %d", n)
}
n = po.pluralForm(4)
if n != 1 {
t.Errorf("Expected 4 for pluralForm(4), got %d", n)
}
n = po.pluralForm(0)
if n != 2 {
t.Errorf("Expected 2 for pluralForm(2), got %d", n)
}
n = po.pluralForm(1000)
if n != 2 {
t.Errorf("Expected 2 for pluralForm(1000), got %d", n)
}
}
func TestTranslationObject(t *testing.T) { func TestTranslationObject(t *testing.T) {
tr := newTranslation() tr := newTranslation()
str := tr.get() str := tr.get()