The default argument applies to the specialization — and, in fact, a specialization must accept (so to speak) the base template’s default argument(s). Attempting to specify a default in the specialization:
template<class A = int, class B=double>
class Base
{};
template<class B=char>
// ...
…is an error.
Likewise, if we change the specialization so that its specialization is for a type other than the default provided by the base template:
template<class A = int, class B=double>
class Base
{};
template<class B>
class Base <char, B>
…then the base template will be chosen.
So, what’s happening is this: first the types for the template arguments are chosen. In this case (no type specified at instantiation), both types are based on the default template arguments specified in the base template.
Then (as a basically separate step) it carries out an analog of overload resolution on all templates that fit those argument types. As usual for overload resolution, a type that’s specified explicitly is preferred over one that’s specified implicitly, so your specialization (which specified int explicitly) is preferred over the base template (which specified int implicitly).