Commit 68fbcf59 authored by mats Dufberg's avatar mats Dufberg
Browse files

Update master to state of develop

parent e76be44c
......@@ -4,10 +4,12 @@ Build
Build.bat
META.*
MYMETA.*
*.mo
.build/
_build/
cover_db/
blib/
inc/
.lwpcookies
.last_cover_stats
nytprof.out
......
Release history for Zonemaster component Zonemaster-Engine
v4.0.0 2020-11-06 (public release version)
[Breaking changes]
- Remove the DISABLE feature (#736, #454)
[Features]
- Added Norwegian language (#806, #786, #751)
- Updated Connectivity03:
- Add support of RIPE Ris whois for ASN lookup (#802, #592),
however, also see #833
- New messages.
- DNSSEC02 with updated logic (#619, #783)
- Add Zonemaster::Engine::Net::IP::ip_is_ipv4 (#689, #688)
[Fixes]
- Updated/corrected translations/PO files (#832, #824, #827, #823, #809,
#811, #808, #796, #794 #789, #787, #779, #757, #776, #761, #735, #745,
#746)
- Updated/corrected data for unit test (#831, #830)
- Corrected/updated Translator.pm (#826, #817, #805, #775, #766, #798,
#776, #755, #749, #748)
- Made po-files.t use gmake for FreeBSD (#819, #816)
- Made FreeBSD to use gmake at installation (#815)
- Corrections to share/Makefile (#813, #814, #807, #804, #771, #726)
- Updated msgid and message arguments (#799, #713, #795, #788, #792, #790,
#782, #776, #758, #760, #743)
- Updated instructions for translators (#772)
- Updated documentation in Profile.pm (#768)
- Corrected system message (#784, #731)
- Corrected Nameserver::Cache (#778, #324)
- Removed duplicate messages from DNSSEC14 (#770, #769)
- Added START/END messages to test cases (#764, #665)
- Clean-up (#801, #767, #765, #752, #736, #454)
- Corrected Consistency05 (#759, #742)
- Corrected Delegation01 (#760, #743)
v3.1.2 2020-05-22
[Fixes]
......
......@@ -5,9 +5,11 @@ docs/Installation.md
docs/logentry_args.md
docs/Profiles.md
docs/Translation.pod
docs/Translation-translators.md
inc/Module/Install.pm
inc/Module/Install/Base.pm
inc/Module/Install/Can.pm
inc/Module/Install/External.pm
inc/Module/Install/Fetch.pm
inc/Module/Install/Makefile.pm
inc/Module/Install/Metadata.pm
......@@ -56,9 +58,9 @@ share/modules.txt
share/profile.json
share/profile_example_logfilter.json
share/locale/da/LC_MESSAGES/Zonemaster-Engine.mo
share/locale/en/LC_MESSAGES/Zonemaster-Engine.mo
share/locale/fr/LC_MESSAGES/Zonemaster-Engine.mo
share/locale/sv/LC_MESSAGES/Zonemaster-Engine.mo
share/locale/nb/LC_MESSAGES/Zonemaster-Engine.mo
share/Makefile
t/00-load.t
t/asn.data
......@@ -92,7 +94,6 @@ t/profiles/Test-consistency05-only.json
t/profiles/Test-consistency06-only.json
t/profiles/Test-consistency-all.json
t/profiles/Test-delegation-all.json
t/profiles/Test-disabled.json
t/profiles/Test-dnssec01-only.json
t/profiles/Test-dnssec02-only.json
t/profiles/Test-dnssec03-only.json
......
\.po$
^share/[^/]*\.mo$
\.tar\.gz$
\bMANIFEST\.SKIP$
^\.perlcriticrc$
......
......@@ -26,10 +26,11 @@ requires 'List::MoreUtils' => 0;
requires 'Locale::TextDomain' => 1.20;
requires 'Module::Find' => 0.10;
requires 'Moose' => 2.0401;
requires 'MooseX::Singleton' => 0.30;
requires 'Net::IP' => 1.26;
requires 'Readonly' => 0;
requires 'Text::CSV' => 0;
requires 'Zonemaster::LDNS' => 2.0;
requires 'Zonemaster::LDNS' => 2.1;
test_requires 'Pod::Coverage' => 0;
test_requires 'Test::Differences' => 0;
......@@ -37,14 +38,27 @@ test_requires 'Test::Exception' => 0;
test_requires 'Test::Fatal' => 0;
test_requires 'Test::Pod' => 1.22;
# Make all platforms include inc/Module/Install/External.pm
requires_external_bin 'find';
if ($^O eq "freebsd") {
requires_external_bin 'gmake';
};
recommends 'Readonly::XS' => 0;
recommends 'Net::IP::XS' => 0;
sub MY::postamble {
return <<'MAKE_FRAG';
pure_all :: share/Makefile
cd share && $(MAKE) touch-po all
MAKE_FRAG
my $text;
if ($^O eq "freebsd") {
# Make FreeBSD use gmake for share/Makefile
$text = 'GMAKE = "gmake"' . "\n"
. 'pure_all :: share/Makefile' . "\n"
. "\t" . 'cd share && $(GMAKE) touch-po all' . "\n";
} else {
$text = 'pure_all :: share/Makefile' . "\n"
. "\t" . 'cd share && $(MAKE) touch-po all' . "\n";
};
return $text;
};
install_share;
......
......@@ -37,7 +37,7 @@ a new language, implementing a new test and the log entries under the directory
## Participation, Contact and Bug reporting
For [participation], [contact] and [bug reporting], please see
For participation, contact and bug reporting, please see
[Zonemaster/README.md](https://github.com/zonemaster/zonemaster/blob/master/README.md).
......@@ -45,8 +45,5 @@ For [participation], [contact] and [bug reporting], please see
The software is released under the 2-clause BSD license. See separate LICENSE file.
[participation]: https://github.com/zonemaster/zonemaster/blob/master/README.md#participation
[contact]: https://github.com/zonemaster/zonemaster/blob/master/README.md#contact
[bug reporting]: https://github.com/zonemaster/zonemaster/blob/master/README.md#bug-reporting
......@@ -69,8 +69,7 @@ If this flag is false, UDP queries that get responses with the C<TC> flag set wi
=head1 POLICY
The policy data is also a nested hash. The top-level keys are upper-case-only versions of test module names, and under them is all the policy data
for that particular module. The keys in the next level down are, with one exception, logger tags. The exception is the special key C<DISABLED>,
which if given a true value will prevent the module from being executed.
for that particular module. The keys in the next level down are, with one exception, logger tags.
The values for the tag keys should be the severity level for that tag. For example, under the C<BASIC> top-level key (for the 'Basic' test module),
there is a key C<NO_GLUE>. That tag will be emitted by the code in the L<Zonemaster::Test::Basic> module if it fails to get NS records for the zone
being tested from its parent zone. Since that is an error that prevents further testing, the message is given the severity level C<CRITICAL>.
......
......@@ -43,7 +43,7 @@ This instruction covers the following operating systems:
```
4) Install binary packages:
```sh
sudo yum install cpanminus libidn-devel openssl-devel perl-Clone perl-core perl-Devel-CheckLib perl-File-ShareDir perl-File-Slurp perl-libintl perl-IO-Socket-INET6 perl-JSON-PP perl-List-MoreUtils perl-Module-Find perl-Moose perl-Net-IP perl-Pod-Coverage perl-Test-Differences perl-Test-Exception perl-Test-Fatal perl-Test-Pod perl-Text-CSV perl-YAML perl-MailTools
```
......@@ -51,7 +51,7 @@ This instruction covers the following operating systems:
5) Install packages from CPAN:
```sh
sudo cpanm Locale::Msgfmt Module::Install Module::Install::XSUtil Test::More
sudo cpanm Email::Valid Locale::Msgfmt Module::Install Module::Install::XSUtil MooseX::Singleton Test::More
```
6) Install Zonemaster::LDNS and Zonemaster::Engine for *CentOS 7*:
......@@ -74,33 +74,25 @@ This instruction covers the following operating systems:
### Installation on Debian
1) Refresh the package information
1) Upgrade to latest patch level
```sh
sudo apt update
sudo apt update && sudo apt upgrade
```
2) Upgrade to latest patch level on Ubuntu 18.04 to get support for DNSSEC algorithm 15 (Ed25519)
* On Ubuntu 18.04:
```sh
sudo apt upgrade
```
3) Install dependencies from binary packages:
2) Install dependencies from binary packages:
```sh
sudo apt install autoconf automake build-essential cpanminus libclone-perl libdevel-checklib-perl libemail-valid-perl libfile-sharedir-perl libfile-slurp-perl libidn11-dev libintl-perl libio-socket-inet6-perl libjson-pp-perl liblist-moreutils-perl liblocale-msgfmt-perl libmodule-find-perl libmodule-install-xsutil-perl libmoose-perl libnet-ip-perl libpod-coverage-perl libreadonly-xs-perl libssl-dev libtest-differences-perl libtest-exception-perl libtest-fatal-perl libtest-pod-perl libtext-csv-perl libtool m4
sudo apt install autoconf automake build-essential cpanminus libclone-perl libdevel-checklib-perl libemail-valid-perl libfile-sharedir-perl libfile-slurp-perl libidn11-dev libintl-perl libio-socket-inet6-perl libjson-pp-perl liblist-moreutils-perl liblocale-msgfmt-perl libmodule-find-perl libmodule-install-xsutil-perl libmoose-perl libmoosex-singleton-perl libnet-ip-perl libpod-coverage-perl libreadonly-xs-perl libssl-dev libtest-differences-perl libtest-exception-perl libtest-fatal-perl libtest-pod-perl libtext-csv-perl libtool m4
```
4) Install dependencies from CPAN:
3) Install dependencies from CPAN:
```sh
sudo cpanm Module::Install Test::More
```
5) Install Zonemaster::LDNS and Zonemaster::Engine.
4) Install Zonemaster::LDNS and Zonemaster::Engine.
* On Debian 10 (Buster):
......@@ -169,7 +161,7 @@ This instruction covers the following operating systems:
* On all versions of FreeBSD install:
```sh
pkg install libidn p5-App-cpanminus p5-Clone p5-Devel-CheckLib p5-Email-Valid p5-File-ShareDir p5-File-Slurp p5-IO-Socket-INET6 p5-JSON-PP p5-List-MoreUtils p5-Locale-libintl p5-Locale-Msgfmt p5-Module-Find p5-Module-Install p5-Module-Install-XSUtil p5-Moose p5-Net-IP-XS p5-Pod-Coverage p5-Readonly-XS p5-Test-Differences p5-Test-Exception p5-Test-Fatal p5-Test-Pod p5-Text-CSV net-mgmt/p5-Net-IP
pkg install devel/gmake libidn p5-App-cpanminus p5-Clone p5-Devel-CheckLib p5-Email-Valid p5-File-ShareDir p5-File-Slurp p5-IO-Socket-INET6 p5-JSON-PP p5-List-MoreUtils p5-Locale-libintl p5-Locale-Msgfmt p5-Module-Find p5-Module-Install p5-Module-Install-XSUtil p5-Moose p5-MooseX-Singleton p5-Net-IP-XS p5-Pod-Coverage p5-Readonly-XS p5-Test-Differences p5-Test-Exception p5-Test-Fatal p5-Test-Pod p5-Text-CSV net-mgmt/p5-Net-IP
```
* On FreeBSD 11.x (11.3 or newer) also install OpenSSL 1.1.1 or newer:
......
# Instructions for translators
See the general [Translation] document for overall documentation on
translation.
### Table of contents
* [Software preparation]
* [Background]
* [Github preparation]
* [Tools]
* [Clone preparation]
* [Translation steps]
* [Adding a new language]
## Software preparation
For the steps below you need to work on a computer with Git, Perl and Gettext.
Select what OS you want to work on from the list below. Other OSs will also work,
but you will have to find instructions elsewhere.
* CentOS
To be written.
* Debian
Install the following:
```
apt install gettext git liblocale-po-perl
```
* FreeBSD
Install the following:
```
pkg install devel/gettext-tools devel/git-lite devel/gmake devel/p5-Locale-PO
```
* Ubuntu
Install the following:
```
apt install gettext git make liblocale-po-perl
```
## Background
The first step in updating the translations is to generate a new template file
("Zonemaster-Engine.pot"). In practice you rarely need to think about generating
it as it is generally performed as an implicit intermediate step.
If you do want to generate it, the command is `make extract-pot` (`gmake
extract-pot` on FreeBSD).
The translated strings are maintained in files named "<LANG-CODE>.po".
Currently there are the following files (and languages):
* da.po (Danish)
* fr.po (French)
* nb.po (Norwegian)
* sv.po (Swedish)
Execute `make xx.po` (`gmake xx.po` on FreeBSD) to update the *PO* file for
language "xx".
Choose the language code for the language that you want to update.
The command will update the *PO* file with new message ids (*msgid*) from the
source code.
This should only be strictly necessary to do when a module has been added,
changed or removed, but it it recommended to do this step every time.
Execute `make update-po` (`gmake update-po` on FreeBSD) to update all the *PO*
files with new message ids from the source code.
By default the updated *PO* file will suggested translations for new message
ids based on fuzzy matching of similar strings. This is not always desirable
and you can disable fuzzy matching by executing one of the following
commands instead:
```
make xx.po MSGMERGE_OPTS=--no-fuzzy-mathing
make update-po MSGMERGE_OPTS=--no-fuzzy-mathing
```
On FreeBSD do this instead:
```
gmake xx.po MSGMERGE_OPTS=--no-fuzzy-mathing
gmake update-po MSGMERGE_OPTS=--no-fuzzy-mathing
```
## Github preparation
For full integration with Zonemaster translation you need a Github account
and a fork of *Zonemaster-Engine*. If you do not have a Github account you
can easily create one at [Github]. If you are not prepared to
create one, contact the Zonemaster work group for instructions by sending
an email to "zonemaster@zonemaster.net".
To create a fork of *Zonemaster-Engine*:
1. Go to [Zonemaster-Engine repository].
2. Make sure you are logged in at Github.
3. Press the "Fork" button.
Make sure that your public *ssh* key is uploaded to Github and that its
private key is available on the computer you are going to work from.
## Tools
The *PO* file can be edited with a plain text editor, but then it is
important to keep the database structure of the file. There are tools that
makes editing of the *PO* files easier. When using those, the *PO* file is
handled as a database instead of as a plain file.
* There is an [add-on to Emacs][emacs PO-mode], which makes updating
and searching in the ".po" file easier and more robust.
* There is also "[GNOME Translation Editor]", a graphical PO editor
available for at least Windows and Linux.
* There are more tools available, either cloud services or programs
for download, and they could be found by searching for "po editor".
## Clone preparation
You need a local clone of the repository to work in.
* Clone the Zonemaster-Engine repository, unless you already
have a clone that you could reuse:
```
git clone https://github.com/zonemaster/zonemaster-engine.git
```
* Enter the directory of the clone created above or already
existing clone:
```
cd zonemaster-engine
```
* If you already have an old clone of Zonemaster-Engine,
run an update.
```
git fetch --all
```
* Now it is time to connect your own fork of *Zonemaster-Engine*
at Github to the created clone, unless you have alreday done that,
in case you can skip the next step.
* You have a user name at Github. Here we use "xxxx" as your user name
and also the name of the remote in clone on the local machine.
```
git remote add xxxx git@github.com:xxxx/zonemaster-engine.git
git fetch --all
```
## Translation steps
The steps in this section will work for most translation work. We
welcome comments on these.
* Check-out the *develop* branch and create a new branch to work in.
You can call the new branch whatever you want, but here we use
the name "translation-update". If that name is already taken,
you have to give it a new name or remove the old branch.
```
git checkout origin/develop
git checkout -b translation-update
```
* Go to the *share* directory and run the update command for the *PO* file
for the language you are going to work with. Replace "xx" with the
language code in question. This should be done every time.
```
cd share
make xx.po
```
On FreeBSD do this instead:
```
cd share
gmake xx.po
```
* The *PO* file is updated with new *msgids*, if any, and now you can start
working with it.
* Update the *PO* file with the tool of your choice. See above. You can copy
the *PO* file to another computer, edit it there, and then copy it back to
your Zonemaster-Engine clone.
* When doing the update, do not change the *msgid*, only the *msgstr*. The
*msgid* cannot and must not be be update in this process. They are the
links between the Perl module and the *PO* file.
* If you find a *msgid* that needs an update, create an [issue][new issue]
or a pull request to have the message updated in the Perl module. If you
create an issue, always include module and message tag, e.g.
"BASIC:NO_PARENT". And make a suggestion of the new *msgid*.
* Inspect every *fuzzy entry* (tagged with "fuzzy"). Update *msgstr*
if needed and remove the "fuzzy" tag. The "fuzzy" tag must always be removed.
* Search for *untranslated entries* (empty *msgstr*) and add a
translation. At the end of the file there could be *obsolete entries*
(lines starting with "#~") and those could have matching translations,
especially of the *msgid* has been changed.
* Any remaining *obsolete entries* (lines at the end of the file starting
with "#~") could be removed. They serve no purpose anymore.
* Check that the messages arguments in all *msgstr* strings match up with
those in the *msgid* strings.
```
../util/check-msg-args xx.po
```
* When the update is completed, it is time to commit the changes. You should
only commit the "xx.po" file.
```
git commit -m 'Write a description of the change' xx.po
```
* There could be other files changed or added that should not be included.
Run the status command to see them.
```
git status
```
* Other changed files could be reset by a "checkout". This could also
be done before creating the commit.
```
git checkout FILENAME
```
* Added files not needed can just be removed. This could also be done
before the commit.
```
rm FILE-NAME
```
* Now push the local branch you created to your fork at Github.
"translation-update" is name of the branch you created above and
have committed the updates to. Use your Github user name instead of
"xxxx".
```
git push -u xxxx translation-update
```
* Go to your fork at Github, https://github.com/xxxx/zonemaster-engine
and use your Github user name instead of "xxxx".
* Select to create a new "pull request" where the base repository
should be *zonemaster/zonemaster-engine* and the base branch should be
*develop* (not *master*). The "head" should be your fork and "compare"
the same branch as you created above and pushed to your fork,
"translation-update".
* Inspect what Github says that will change by the pull request. It should
only be the *PO* file that you have updated and nothing else. If additional
files are listed, please correct or request for help.
* Press "create pull request", write a nice description and press "create"
again.
* If you go back to your own computer and just keep the clone as it is, you
can easily update the pull request if needed with more changes to the same
*PO* file. When the pull request has been merged by the Zonemaster work group,
you can delete the local clone and on your Github fork you can remove the
branch. Or keep them for next time.
## Adding a new language
If you want to add a new language, then follow the steps above with some
modifications. Before you add a language contact the Zonemaster project
to discuss timeplan and other aspects of the new language. Every language must
be update each time a message is added or changed.
Above you found the following step that must now be modified, but run the
steps before this step:
> * Go to the *share* and update the *PO* file for the language you are going
> to work with. Replace "xx" with the language code in question.
> ```
> cd share
> make xx.po
> ```
>
> On FreeBSD do this instead:
> ```
> cd share
> gmake xx.po
> ```
The new language is not there and cannot be updated. Instead you have to
create the new language file (*PO* file). The easiest way is to make a copy
of an existing file.
* Determine what the language code of the new language should be. I must be
a code that is available in the *locale* system. Try the following commands
to see if it is available. Replace "xx" with that code that you think it
should be.
```
locale -a | grep xx # Works at least in FreeBSD
grep xx /etc/locale.gen # Works at least in Ubuntu 18.04
```
* Go to the *share* and update the *PO* file for some language, say Swedish,
and make a copy of that to the new file name. And then reset the *PO* file
for Swedish.
```
cd share
make sv.po
cp sv.po xx.po
git checkout sv.po
```
On FreeBSD do this instead:
```
cd share
gmake sv.po
cp sv.po xx.po
git checkout sv.po
```
* You have to "add" the new file to git before you start working on it.
```
git add xx.po
```
* When you do the update of the new *PO* file you have to replace all *msgstr*
in Swedish with the translation in the new language, but also update the
"Language" field in the header.
* Now you go back to the steps and continue in the same was with an existing
language.
[Adding a new language]: #adding-a-new-language
[Background]: #background
[Clone preparation]: #clone-preparation
[Emacs PO-mode]: https://www.gnu.org/software/gettext/manual/html_node/PO-Mode.html#PO-Mode
[GNOME Translation Editor]: https://wiki.gnome.org/Apps/Gtranslator
[Github preparation]: #github-preparation
[Github]: https://github.com/
[Software preparation]: #software-preparation
[Tools]: #tools
[Translation]: https://github.com/zonemaster/zonemaster-engine/blob/develop/docs/Translation.pod
[Translation steps]: #translation-steps
[Zonemaster-Engine repository]: https://github.com/zonemaster/zonemaster-engine
[new issue]: https://github.com/zonemaster/zonemaster-engine/issues/new
......@@ -11,6 +11,11 @@ All translation files live in the F<share> directory in the
L<Zonemaster::Engine> source directory and all commands described here are
executed from that directory.
=head2 For translators
Instructions for translators can be found in
L<https://github.com/zonemaster/zonemaster-engine/blob/develop/docs/Translation-translators.md>
=head2 For developers of Zonemaster test modules
The test module code should produce log messages with message tags, as documented
......@@ -44,179 +49,13 @@ Every time you add, remove or modify a tag or its message id, re-run
C<make update-po> as descibed in the next section.
Make sure the message tag comments are properly added and up to date.
=head2 For translators
=head3 Software preparation
For the steps below you need to work on a computer with Git, Perl and Xgettext.
Select what OS you want to work on. Other OSs will also work, but you will
have to find instructions elsewhere.
=head4 FreeBSD
Install the following:
C<pkg install git-lite devel/p5-Locale-XGettext devel/p5-Locale-Msgfmt devel/gmake>
=head4 CentOS
To be written.
=head4 Ubuntu
To be written.
=head4 Debian
To be written
=head3 Background
The first step in updating the translations is to generate a new template file
(F<Zonemaster-Engine.pot>).
In practice you rarely need to think about generating it as it is generally
performed as an implicit intermediate step.
If you do want to generate it, the command is C<make extract-pot>
(C<gmake extract-pot> on FreeBSD).