Existing deep learning approaches have achieved high performance in encrypted network traffic analysis tasks. However, some realistic scenarios, such as open-set recognition on dynamically changing tasks, challenge previous methods. Classic few-shot learning methods are used widely for these tasks in certain domains, such as computer vision and natural language processing. Nonetheless, few-shot open-set recognition for encrypted network traffic is still an unexplored area. This paper proposes a probability based task adaptive Siamese open-set recognition model for encrypted network traffic classification. Our contributions are threefold: First, we introduce generated positive and negative pairs into the Siamese Neural Network training process to shape a more precise similarity boundary through bidirectional dropout data augmentation. Second, we utilize Dirichlet Process Gaussian Mixture Model (DPGMM) distribution to fit the similarity scores of the negative pairs constructed by the support set of each query task, and create a new open- set recognition metric. Third, by leveraging the extracted features from coarse and fine-granular levels, we construct a hierarchical cross entropy loss to improve the confidence of the similarity score. Extensive experiments on a network traffic dataset and the Omniglot dataset demonstrate the superiority of our proposed approaches, which can respectively obtain up to 4.5% and 1.2% performance gain in terms of accuracy as well as 4.0% and 1.8% in terms of area under the receiver operating characteristic (AUROC).