Thursday, December 31, 2020

Playing one sentence from an audio file

 

I’m using Lazarus to develop some tools for myself for language immersion. What I want is to hear the same sentence again and again, until I can fully understand it, I know what it means, I can recognize the individual words (and in which form they are), and I can reproduce it. That takes (depending on the language, the sentence, its length, and level) a lot of repetitions. I usually prefer seven.

DuoLingo is a great app - but you have to press that sound button again and again. Besides that, the voices are artifical. Also in Rosetta Stone you have to press the sound button over and over. Apart from that, Rosetta Stone has the advantage that the voices are human, and that you don’t see the translation (that is usually an advantage, but can in some cases be a drawback too).

Anyway, I’ve a program which plays sentences again and again. I feed the program with sentences from books. I split the book text into sentences (with a home made tool) and I add translations.

Yes, translations: sometimes I need them to understand what the reader is saying, and in most (literary) books a vocabulary is used that exceeds my current level. And the books I use are usually older because I get the voices from sites like archive.org or librevox, and these are books in the public domain.

I’m using deepl.com for translations. Their translations are acceptable to understand what’s going on. There can be errors but I can correct them. Deepl translations are, in my experience, better than translations from yandex or google.

So my program reads aloud sentences again and again, but how? Using bass.dll! That’s a cool library which is free (for personal use) and works on Windows and on Linux (and more), and works with Lazarus (and more). Its documentation is good but lacks some examples. For instance, I want to repeat that sentence over and over...

Playing a sentence (as a part of a large mp3) is possible with bass.dll and Lazarus. You can play from a certain position and add a timer, and stop the sound on the timer event. But timers are in general cumbersome and imprecise. It works most of the time but... if you’re computer is busy with some other task, you sometimes hear the start of the next sentence. And even if that’s just the start (say a “ST”), it’s annoying. On Linux, if I run it on my Windows computer in a virtual box, it’s even worse: most of the time it exceeds the scheduled time. So a timer can’t be used for this purpose.

Bass.dll offers another way to play a part of an mp3 and that is a sync (BASS_ChannelSetSync). That works but... if I also get sound levels in between (and I want that too), it often crashes. It apparently can’t do that during the loop back and we can’t exactly predict when that happens. I tried to fix that but I couldn’t, so I gave up. Besides this, the code was more complex because of the callbacks involved.

But there are more possibilities. I found two ways which both work. The first way is to write a WAV file either on memory or even on disk, and play that WAV. There is a source code example converting to WAV (to disk) so that was easy to adapt, and in memory it works the same way. The second way is to use a bass “sample” and play that back. There were no complete examples of that, but combining some other examples I got it working. This is the method I finally selected because it is, code wise, simple, and precise (without any timer involved) and reliable (no crashes observed).

A minimal sample doing this is presented here below and I hope it is useful for anyone who wants to do a similar thing. The code here below is Lazarus (free pascal) code, but it will be straightforward to use it in another language.

procedure PlayPart;
const bufferSize = 10000;
var
inputChannel: HSTREAM;
info: BASS_CHANNELINFO;
outputChannel : HSTREAM;
sample : HSAMPLE;

buffer : array [1..bufferSize] of byte;
b1, b2 : QWORD;
bytesRead : DWORD;
memo : TMemoryStream;

begin
// You should open the channel in the decode mode
inputChannel := BASS_StreamCreateFile(false, PChar('c:\data\books\Gaidar\DrummersFate\audio\01.mp3'), 0, 0, BASS_STREAM_DECODE);
if inputChannel <= 0 then exit;

BASS_ChannelGetInfo(inputChannel, info);

// Specify here the begin and end times
b1 := BASS_ChannelSeconds2Bytes(inputChannel, 223.0);
b2 := BASS_ChannelSeconds2Bytes(inputChannel, 225.33);

BASS_ChannelSetPosition(inputChannel, b1, BASS_POS_BYTE);

memo := TMemoryStream.Create;
try
while (BASS_ChannelIsActive(inputChannel) = 1) and (b1 < b2) do
begin
bytesRead := BASS_ChannelGetData(inputChannel, @buffer, min(bufferSize, b2 - b1));
memo.Write(buffer, bytesRead);
inc(b1, bytesRead);
end;
BASS_StreamFree(inputChannel);

// Create a sample, with the input sound specifications, and max one simultaneous playback.
// Specify a flag to automatically repeat it.
// If that is not necessary, a flag of 0 is fine for this example.
sample := BASS_SampleCreate(memo.size, info.freq, info.chans, 1, BASS_SAMPLE_LOOP);

// Fill it with the read data
BASS_SampleSetData(sample, memo.memory);
finally
memo.free;
end;

// Create a channel (this is required) and play it.
outputChannel := BASS_SampleGetChannel(sample, true);
BASS_ChannelPlay(outputChannel, true);
end;

This is the whole code (except for initialization of the bass library itself, but that should always be done, once, and is straightforward).

It will play this sentence:



As far as I know, it's not possible to save this part as an mp3 (as published here). But you can save it as a WAV file. There are numerous tools to extract parts from a large mp3 and save them. What I needed was to play a part, with software. And for that I use bass and Lazarus.

Wednesday, December 31, 2014

Vectorization with square point buffers

Vectorization with square point buffers


This year (2014) the buffer algorithm of Boost.Geometry was finally released in Boost 1.56

With buffer we mean a zone around a geometry. This is the GIS term, also used by OGC. In other environments it is often known as inflate / deflate, or shrink and expand, or Minkowski sum.

In GIS the buffer (with a positive distance) creates a larger version of e.g. a polygon. If a negative distance is specified, a smaller version is created.

As the documentation shows, various strategies can be used to:
  • control the buffer-distance (either symmetric or asymmetric)
  • control the joins (rounded or miter)
  • control the ends for linestrings (rounded or flat)
  • control the sides (straight)
  • control the zones around points (or degenerated linestrings) (circular or square)
Strategies are provided, but they can also be provided by the library user, e.g. to create some fancy effects.

This short blog gives an example of the strategies for zones around points, using the square version. This strategy has the nice property that can be used to convert grids or images to vectors (polygons).

strategy::buffer::point_square


See also the Wiki page. Pixels are converted to polygons. Suppose we add all centers of all filled pixels (e.g. the black pixels) into a multi point. We then buffer that multi-point with a distance of half the pixel size. And we use the strategy to create square buffers around each point. Then all adjacent gridcells will be connected. A polygon (or a multi-polygon) will be created, which is exactly what we wanted...

Suppose we have the following image (from this source) of a black cat.


Cat (pixels)

We create a multi-point of it, that is done manually:

MULTIPOINT(1 0,2 0,3 0,4 0,5 0,6 0,7 0,   0 1,1 1,2 1,3 1,4 1,5 1,7 1,8 1,  
0 2,1 2,2 2,3 2,4 2,5 2,8 2, 0 3,1 3,2 3,3 3,4 3,5 3,8 3,   0 4,1
4,2 4,3 4,4 4,5 4,8 4, 0 5,1 5,2 5,3 5,4 5,5 5,8 5,   1 6,2 6,3 6,4
6,7 6,8 6,   1 7,2 7,3 7,4 7,7 7, 1 8,2 8,3 8,4 8,7 8,   2 9,3
9,   6 9,7 9,   1 10,2 10,3 10,4 10,   1 11,2 11,3 11,4
11,   1 12,2 12,3 12,4 12,   1 13,4 13)

If we use the next code fragment:

    typedef boost::geometry::model::d2::point_xy<double> point;
    typedef boost::geometry::model::polygon<point> polygon;
    boost::geometry::strategy::buffer::distance_symmetric<double> distance_strategy(0.5);
    boost::geometry::strategy::buffer::join_round join_strategy;
    boost::geometry::strategy::buffer::end_round end_strategy;
    boost::geometry::strategy::buffer::point_square point_strategy;
    boost::geometry::strategy::buffer::side_straight side_strategy;
    boost::geometry::model::multi_polygon<polygon> result;
    boost::geometry::model::multi_point<point> mp;
    boost::geometry::read_wkt(cat, mp); // the cat std::string is somewhere defined
    boost::geometry::buffer(mp, result,
                distance_strategy, side_strategy,
                join_strategy, end_strategy, point_strategy);


The resulting polygon will look like this:

Cat (polygon)


Sunday, December 30, 2012

Preludes in all 24 major and minor keys

Preludes in all 24 major and minor keys

Many composers in classical music have written sets of preludes in all major and minor, so 24, keys. The first well-known composer who wrote such a set was J.S. Bach, with his Well-Tempered Clavier. He wrote two sets, and each prelude is followed by a fuga. So 2*24*2=96 pieces. After him, at least 20 composers created similar sets (though often without fuga's) for piano, in all 24 keys. This blog links three sets, the prelude sets of Chopin, Scriabin and Kapustin.

Chopin

Chopin's well-known set is Opus 28, 24 preludes for piano solo. The preludes are in the order of the circle of fifths, with major and minor keys alternating. The preludes which are linked below are played by Arthur Rubinstein.

Interesting with this sequence is that all preludes in major are numbered odd, and all preludes in minor are numbered even. Also note that, because of this and the properties of the circle of fifths, the number of sharps/flats is equal for all pairs. So 1 and 2 have no sharps nor flats, 3 and 4 have one sharp, 5 and 6 have two, this is increasing until 13 and 14 (having either 6 flats or 6 sharps), then it switches to 5 flats for 15 and 16, this number is decreasing, until finally 23 and 24 have one flat.

Scriabin

Scriabin's set is Opus 11, 24 preludes for piano solo. He used exactly the same sequence as Chopin did. Number 5 was written in Amsterdam, in 1896. Number 8 is magnificently (and slowly) played by his friend Rachmaninov (!), here. Below are links to all preludes, played by Sofronitsky, a famous Scriabin interpreter and also his son-in-law. Scriabin started another set but did not finish that set. Preludes of this unfinished set are spread over Opus 13, 15, 16, and 17.

Kapustin

Kapustin's set is Opus 53, 24 preludes for piano solo. Kapustin used the same sequence as Chopin and Scriabin. These preludes are written in 1989, and denoted as in Jazz Style. Writing Jazz idioms in classical forms is Kapustin's speciality. Kapustin's set of preludes is marvelous. Below you find youtube links to all 24 preludes, played by the composer himself, Nikolai Kapustin. Besides this Opus 53, Kapustin wrote 20 piano sonata's, 6 piano concerto's, and much more. He is on the way to become the best componist of the 20th or 21th century! But still unknown to most people. Kapustin did also write another set of 24 preludes and fuga's in all keys, Opus 82. This set is not discussed here (because of similarity - it is more similar to Bach's Well-Tempered Clavier).

I could have added more sets, for example, I like for example Rachmaninov too. But the sets below are quite similar, using the same sequence, with composer names all ending with "in" :-), so I finished it this way. The images of the circles of fifths are a bit inspired on the Wiki source, but for the rest created from scratch for this blog page.

Enjoy the music, 72 preludes.

Sunday, December 4, 2011

SQLite, Ado.Net, SpatiaLite


SQLite, Ado.Net, SpatiaLite


SQLite is a small but great database. SpatiaLite is its spatial extension. ADO.NET is the method to access databases in .NET.

This blog shows how to combine them, because I did found all information on the web but not a complete and up-to-date story.


SQLite and Ado.Net

An introduction (up and running in 3 minutes) is here. It is good but not up-to-date. In the meantime the ADO.NET driver is transferred from sourceforge/phxsoftware to sqlite.org, which is of course a good sign.

There is only one piece of software which you really need and that is the ADO.NET driver. Furthermore there are some optional (but very useful tools) which I describe below.


ADO.NET driver

The current ADO.NET driver page is here, and more specificly you can download the drivers from this page. Download for example the sqlite-netFx40-setup-bundle-x86-2010-1.0.77.0.exe (10.25 MiB)  (at least I did) and install it. The installation folder will contain the driver, and also a manual and a sample database (northwindEF.db).

Setting up the project

Create a new project (or use an existing) in C# Express or Visual Studio (I'm using 2010). I created a Console Project.

Most important is of course to add a reference. Typically you need  the System.Data.SQLite.dll from the installation folder (typically "c:\Program Files (x86)\System.Data.SQLite\2010\bin" ). You can either reference it from there or copy it to your project folder and reference it.

Writing the code.

See also the original 3 minutes introduction for some code. Here a different sample is presented. I like complete code and I also include some extra optional pieces which I needed (creating a new database from scratch, and turning on foreign keys).

So a complete program looks like:
// Copyright (c) 2011 Barend Gehrels 
using System;
using System.Data.Common;
using System.Data.SQLite;

namespace SQLiteAndAdoDotNet
{
    class Program
    {
        const string mydb = @"c:\temp\mydatabase.db";

        static void Main(string[] args)
        {
            bool isNew = false;

            if (!System.IO.File.Exists(mydb))
            {
                // Create a new SQLite database if it does not exist
                SQLiteConnection.CreateFile(mydb);
                isNew = true;
            }

            using (SQLiteConnection connection = new SQLiteConnection(@"Data Source=" + mydb))
            {
                connection.Open();

                // Recommended action for any SQLite database
                ExecuteStatement("PRAGMA foreign_keys = ON", connection);

                if (isNew)
                {
                    // Create a table for a new database
                    ExecuteStatement(@"
                        create table mytable
                            (
                                id integer primary key autoincrement,
                                myfield text,
                                mydate integer -- dates can be stored as either text, real or integer
                            )
                        "
, connection);
                }

                // Insert something
                ExecuteStatement("insert into mytable(myfield,mydate) values('my row',date('now'))",
                    connection);

                // Retrieve something
                using (SQLiteCommand command = new SQLiteCommand("select *,datetime(mydate) as dt_ok from mytable",
                            connection))
                {
                    using (DbDataReader reader = command.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            System.Console.WriteLine(String.Format(
                                "Row id={0} field={1} date={2}, dt={3}",
                                reader["id"], reader["myfield"],
                                reader["mydate"], reader["dt_ok"]));
                        }
                    }
                }
            }
            System.Console.WriteLine("Done");
        }

        private static void ExecuteStatement(string statement, SQLiteConnection connection)
        {
            using (SQLiteCommand command = new SQLiteCommand(statement, connection))
            {
                command.ExecuteNonQuery();
            }
        }
    }
}

I factored out one method to avoid repetitions. In another blog I'll show you a different approach.

The foreign key pragma is of course not necessary in this short sample. But in a "real" database, with a datamodel and foreign keys, you will need it. Otherwise cascaded deletes will fail unexpectedly!

One caveat

Beware for datetime fields, stored as real or integer! SQLite stores date types in any field, which is reasonable. But if you select such a date(time) field in the normal way you will get only the year back! So alas you have to cast it with the SQLite datetime function (described here). Dates stored as text do not have this caveat.

Useful tools

I like the SQLite Studio which can be downloaded freely from here. You can hardly without such a tool and this one is really cool. If you are coming from SQL Server you might also like a converter, and this one is good, free, and with source.

Using SpatialLite

Using SpatialLite is pretty much the same, from ADO.NET perspective. But of course you have to download some extra libraries. The SpatialLite library of course, but you also need at least two extra libraries. So here we go (see also StackOverflow):
  • download the SpatiaLite DLL from this page (libspatialite-win-x86-2.3.1.zip)
  • download proj-win as well from the same page (proj-win-x86-4.6.1.zip)
  • download iconv as well from the same page (libiconv-win-x86-1.9.2.zip)
  • download geos as well from the same page (geos-win-x86-3.1.1.zip) (I added geos later, after the comment of rmales, for me it was not necessary because it was found in my path, via PostGIS.)
But them all into your binary folder (Debug and Release). For our test project we also need a spatial database. You can create one from a shapefile (using the spatialite-gui from the same page) or download a test database from SpatialLite, here. I downloaded and extracted world.7z because I like to see world countries (see my other blogs).

Then, before you have to use it, you have to load the extension. That is pretty easy with an SQL statement, select load_extension('libspatialite-1.dll'). If it finds this DLL and the other two, it will run fine (and otherwise it will mention which is missing). Alas I could not get it working in sqlitestudio.

Then below is the program I used, and it runs fine. It is quite similar to the one above but I prefer to list it completely again. In next blog(s) we will do other interesting and maybe surprising things with it.
// Copyright (c) 2011 Barend Gehrels 
using System;
using System.Data.Common;
using System.Data.SQLite;

namespace SQLiteAndAdoDotNet
{
    class Program
    {
        const string mydb = @"c:\bdata\blogs\data\world.sqlite";

        static void Main(string[] args)
        {
            using (SQLiteConnection connection = new SQLiteConnection(@"Data Source=" + mydb))
            {
                connection.Open();

                // Recommended action for any SQLite database
                ExecuteStatement("PRAGMA foreign_keys = ON", connection);

                // Required action for any SpatiaLite database
                ExecuteStatement("select load_extension('libspatialite-1.dll')", connection);

                // Retrieve some cities in Europe
                using (SQLiteCommand command = new SQLiteCommand(
                        @"
                            select wup_aggl,AsText(Geometry) as wkt,
                                X(Geometry) as x,Y(Geometry) as y
                            from Cities
                            where MBRContains(BuildMBR(-2,40,10,80),Geometry) = 1
                        "
, connection))
                {
                    using (DbDataReader reader = command.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            System.Console.WriteLine(String.Format(
                                "City: {0} location: {1}, {2} wkt: {3}",
                                reader["wup_aggl"], reader["x"], reader["y"], reader["wkt"]));
                        }
                    }
                }
            }
            System.Console.WriteLine("Done");
        }

        private static void ExecuteStatement(string statement, SQLiteConnection connection)
        {
            using (SQLiteCommand command = new SQLiteCommand(statement, connection))
            {
                command.ExecuteNonQuery();
            }
        }

    }
}


Useful tools

Of course you can use Quantum GIS to perfectly visualize your SpatiaLite database. The SpatiaLite page also contains many useful links.

Summary

Above two complete programs to use SQLite and/or SpatiaLite in your .NET application. This is the C# version, VB.NET will work either.


Saturday, November 19, 2011

Linestring/polygon intersection


Linestring/polygon intersection


After a long while a blog again, a short one this time. I'm currently implementing intersections linestring/polygon for Boost.Geometry. Well, the basics (calculating intersection points) are already there for a long time, but for this combination the intersection points should be followed in another way and the correct pieces should be outputted.

When I do these things I normally check the results with both PostGIS and SQL Server. However, I'm encountering something weird in both of them.

The testcase

My testcase is looking like this:




SQL Server

In SQL Server the intersection is correctly done, but the linestring is reversed! I'm using this query:
with viewy as
(
   select
      geometry::STGeomFromText('POLYGON((1 1,1 3,3 3,3 1,1 1))', 0) as p,
      geometry::STGeomFromText('LINESTRING(2 2,1 2,1 3,2 3)', 0) as q
)
select
   p.STIntersection(q).STLength() as len,
   p.STIntersection(q).STAsText() as wkt
from viewy;

and this is my output:

3    LINESTRING (2 3, 1 3, 1 2, 2 2)

You see, the linestring is reversed! It is not only reversed with this linestring, but with all linestrings. Even if they are, for example, complete inside the polygon. If I reverse the polygon (from clockwise to counter clockwise), the output is still reversed. Mysterious...

PostGIS

Also a surprise in PostGIS.The query is looking there as following:

with viewy as
(
   select
      ST_GeomFromText('POLYGON((1 1,1 3,3 3,3 1,1 1))') as p,
      ST_GeomFromText('LINESTRING(2 2,1 2,1 3,2 3)') as q
)
select
   ST_Length(ST_Intersection(p, q)) as len,
   ST_AsText(ST_Intersection(p, q)) as wkt
from
viewy;
and this is here the result:

3;"MULTILINESTRING((1 2,1 3),(1 3,2 3),(2 2,1 2))"

I'm getting a multilinestring back! With respect to contents, it is OK. But it is surprising...

OGC

I did not look it up but I don't think this specific behaviour is specified. So yes, all implementations will be correct but...

Boost.Geometry

At the moment of writing, technically more difficult cases as this is one are not completely finished. But I can already tell that the linestring will not be reversed and that the output will consist of only one linestring...