-
Notifications
You must be signed in to change notification settings - Fork 822
Expand file tree
/
Copy pathspeech_to_text_v1.py
More file actions
9556 lines (8720 loc) · 466 KB
/
speech_to_text_v1.py
File metadata and controls
9556 lines (8720 loc) · 466 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# coding: utf-8
# (C) Copyright IBM Corp. 2015, 2025.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# IBM OpenAPI SDK Code Generator Version: 3.105.0-3c13b041-20250605-193116
"""
The IBM Watson™ Speech to Text service provides APIs that use IBM's
speech-recognition capabilities to produce transcripts of spoken audio. The service can
transcribe speech from various languages and audio formats. In addition to basic
transcription, the service can produce detailed information about many different aspects
of the audio. It returns all JSON response content in the UTF-8 character set.
The service supports two types of models: previous-generation models that include the
terms `Broadband` and `Narrowband` in their names, and next-generation models that include
the terms `Multimedia` and `Telephony` in their names. Broadband and multimedia models
have minimum sampling rates of 16 kHz. Narrowband and telephony models have minimum
sampling rates of 8 kHz. The next-generation models offer high throughput and greater
transcription accuracy.
Effective **31 July 2023**, all previous-generation models will be removed from the
service and the documentation. Most previous-generation models were deprecated on 15 March
2022. You must migrate to the equivalent large speech model or next-generation model by 31
July 2023. For more information, see [Migrating to large speech
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate).{:
deprecated}
For speech recognition, the service supports synchronous and asynchronous HTTP
Representational State Transfer (REST) interfaces. It also supports a WebSocket interface
that provides a full-duplex, low-latency communication channel: Clients send requests and
audio to the service and receive results over a single connection asynchronously.
The service also offers two customization interfaces. Use language model customization to
expand the vocabulary of a base model with domain-specific terminology. Use acoustic model
customization to adapt a base model for the acoustic characteristics of your audio. For
language model customization, the service also supports grammars. A grammar is a formal
language specification that lets you restrict the phrases that the service can recognize.
Language model customization and grammars are available for most previous- and
next-generation models. Acoustic model customization is available for all
previous-generation models.
API Version: 1.0.0
See: https://cloud.ibm.com/docs/speech-to-text
"""
from enum import Enum
from typing import BinaryIO, Dict, List, Optional
import json
from ibm_cloud_sdk_core import BaseService, DetailedResponse
from ibm_cloud_sdk_core.authenticators.authenticator import Authenticator
from ibm_cloud_sdk_core.get_authenticator import get_authenticator_from_environment
from ibm_cloud_sdk_core.utils import convert_list, convert_model
from .common import get_sdk_headers
##############################################################################
# Service
##############################################################################
class SpeechToTextV1(BaseService):
"""The Speech to Text V1 service."""
DEFAULT_SERVICE_URL = 'https://api.us-south.speech-to-text.watson.cloud.ibm.com'
DEFAULT_SERVICE_NAME = 'speech_to_text'
def __init__(
self,
authenticator: Authenticator = None,
service_name: str = DEFAULT_SERVICE_NAME,
) -> None:
"""
Construct a new client for the Speech to Text service.
:param Authenticator authenticator: The authenticator specifies the authentication mechanism.
Get up to date information from https://github.com/IBM/python-sdk-core/blob/main/README.md
about initializing the authenticator of your choice.
"""
if not authenticator:
authenticator = get_authenticator_from_environment(service_name)
BaseService.__init__(self,
service_url=self.DEFAULT_SERVICE_URL,
authenticator=authenticator)
self.configure_service(service_name)
#########################
# Models
#########################
def list_models(
self,
**kwargs,
) -> DetailedResponse:
"""
List models.
Lists all language models that are available for use with the service. The
information includes the name of the model and its minimum sampling rate in Hertz,
among other things. The ordering of the list of models can change from call to
call; do not rely on an alphabetized or static list of models.
**See also:** [Listing all
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-list#models-list-all).
:param dict headers: A `dict` containing the request headers
:return: A `DetailedResponse` containing the result, headers and HTTP status code.
:rtype: DetailedResponse with `dict` result representing a `SpeechModels` object
"""
headers = {}
sdk_headers = get_sdk_headers(
service_name=self.DEFAULT_SERVICE_NAME,
service_version='V1',
operation_id='list_models',
)
headers.update(sdk_headers)
if 'headers' in kwargs:
headers.update(kwargs.get('headers'))
del kwargs['headers']
headers['Accept'] = 'application/json'
url = '/v1/models'
request = self.prepare_request(
method='GET',
url=url,
headers=headers,
)
response = self.send(request, **kwargs)
return response
def get_model(
self,
model_id: str,
**kwargs,
) -> DetailedResponse:
"""
Get a model.
Gets information for a single specified language model that is available for use
with the service. The information includes the name of the model and its minimum
sampling rate in Hertz, among other things.
**See also:** [Listing a specific
model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-list#models-list-specific).
:param str model_id: The identifier of the model in the form of its name
from the output of the [List models](#listmodels) method.
:param dict headers: A `dict` containing the request headers
:return: A `DetailedResponse` containing the result, headers and HTTP status code.
:rtype: DetailedResponse with `dict` result representing a `SpeechModel` object
"""
if not model_id:
raise ValueError('model_id must be provided')
headers = {}
sdk_headers = get_sdk_headers(
service_name=self.DEFAULT_SERVICE_NAME,
service_version='V1',
operation_id='get_model',
)
headers.update(sdk_headers)
if 'headers' in kwargs:
headers.update(kwargs.get('headers'))
del kwargs['headers']
headers['Accept'] = 'application/json'
path_param_keys = ['model_id']
path_param_values = self.encode_path_vars(model_id)
path_param_dict = dict(zip(path_param_keys, path_param_values))
url = '/v1/models/{model_id}'.format(**path_param_dict)
request = self.prepare_request(
method='GET',
url=url,
headers=headers,
)
response = self.send(request, **kwargs)
return response
#########################
# Synchronous
#########################
def recognize(
self,
audio: BinaryIO,
*,
content_type: Optional[str] = None,
model: Optional[str] = None,
speech_begin_event: Optional[bool] = None,
language_customization_id: Optional[str] = None,
acoustic_customization_id: Optional[str] = None,
base_model_version: Optional[str] = None,
customization_weight: Optional[float] = None,
inactivity_timeout: Optional[int] = None,
keywords: Optional[List[str]] = None,
keywords_threshold: Optional[float] = None,
max_alternatives: Optional[int] = None,
word_alternatives_threshold: Optional[float] = None,
word_confidence: Optional[bool] = None,
timestamps: Optional[bool] = None,
profanity_filter: Optional[bool] = None,
smart_formatting: Optional[bool] = None,
smart_formatting_version: Optional[int] = None,
speaker_labels: Optional[bool] = None,
grammar_name: Optional[str] = None,
redaction: Optional[bool] = None,
audio_metrics: Optional[bool] = None,
end_of_phrase_silence_time: Optional[float] = None,
split_transcript_at_phrase_end: Optional[bool] = None,
speech_detector_sensitivity: Optional[float] = None,
sad_module: Optional[int] = None,
background_audio_suppression: Optional[float] = None,
low_latency: Optional[bool] = None,
character_insertion_bias: Optional[float] = None,
**kwargs,
) -> DetailedResponse:
"""
Recognize audio.
Sends audio and returns transcription results for a recognition request. You can
pass a maximum of 100 MB and a minimum of 100 bytes of audio with a request. The
service automatically detects the endianness of the incoming audio and, for audio
that includes multiple channels, downmixes the audio to one-channel mono during
transcoding. The method returns only final results; to enable interim results, use
the WebSocket API. (With the `curl` command, use the `--data-binary` option to
upload the file for the request.)
**See also:** [Making a basic HTTP
request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-http#HTTP-basic).
### Streaming mode
For requests to transcribe live audio as it becomes available, you must set the
`Transfer-Encoding` header to `chunked` to use streaming mode. In streaming mode,
the service closes the connection (status code 408) if it does not receive at
least 15 seconds of audio (including silence) in any 30-second period. The service
also closes the connection (status code 400) if it detects no speech for
`inactivity_timeout` seconds of streaming audio; use the `inactivity_timeout`
parameter to change the default of 30 seconds.
**See also:**
* [Audio
transmission](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-input#transmission)
*
[Timeouts](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-input#timeouts)
### Audio formats (content types)
The service accepts audio in the following formats (MIME types).
* For formats that are labeled **Required**, you must use the `Content-Type`
header with the request to specify the format of the audio.
* For all other formats, you can omit the `Content-Type` header or specify
`application/octet-stream` with the header to have the service automatically
detect the format of the audio. (With the `curl` command, you can specify either
`"Content-Type:"` or `"Content-Type: application/octet-stream"`.)
Where indicated, the format that you specify must include the sampling rate and
can optionally include the number of channels and the endianness of the audio.
* `audio/alaw` (**Required.** Specify the sampling rate (`rate`) of the audio.)
* `audio/basic` (**Required.** Use only with narrowband models.)
* `audio/flac`
* `audio/g729` (Use only with narrowband models.)
* `audio/l16` (**Required.** Specify the sampling rate (`rate`) and optionally the
number of channels (`channels`) and endianness (`endianness`) of the audio.)
* `audio/mp3`
* `audio/mpeg`
* `audio/mulaw` (**Required.** Specify the sampling rate (`rate`) of the audio.)
* `audio/ogg` (The service automatically detects the codec of the input audio.)
* `audio/ogg;codecs=opus`
* `audio/ogg;codecs=vorbis`
* `audio/wav` (Provide audio with a maximum of nine channels.)
* `audio/webm` (The service automatically detects the codec of the input audio.)
* `audio/webm;codecs=opus`
* `audio/webm;codecs=vorbis`
The sampling rate of the audio must match the sampling rate of the model for the
recognition request: for broadband models, at least 16 kHz; for narrowband models,
at least 8 kHz. If the sampling rate of the audio is higher than the minimum
required rate, the service down-samples the audio to the appropriate rate. If the
sampling rate of the audio is lower than the minimum required rate, the request
fails.
**See also:** [Supported audio
formats](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formats).
### Large speech models and Next-generation models
The service supports large speech models and next-generation `Multimedia` (16
kHz) and `Telephony` (8 kHz) models for many languages. Large speech models and
next-generation models have higher throughput than the service's previous
generation of `Broadband` and `Narrowband` models. When you use large speech
models and next-generation models, the service can return transcriptions more
quickly and also provide noticeably better transcription accuracy.
You specify a large speech model or next-generation model by using the `model`
query parameter, as you do a previous-generation model. Only the next-generation
models support the `low_latency` parameter, and all large speech models and
next-generation models support the `character_insertion_bias` parameter. These
parameters are not available with previous-generation models.
Large speech models and next-generation models do not support all of the speech
recognition parameters that are available for use with previous-generation models.
Next-generation models do not support the following parameters:
* `acoustic_customization_id`
* `keywords` and `keywords_threshold`
* `processing_metrics` and `processing_metrics_interval`
* `word_alternatives_threshold`
**Important:** Effective **31 July 2023**, all previous-generation models will be
removed from the service and the documentation. Most previous-generation models
were deprecated on 15 March 2022. You must migrate to the equivalent large speech
model or next-generation model by 31 July 2023. For more information, see
[Migrating to large speech
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate).
**See also:**
* [Large speech languages and
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-large-speech-languages)
* [Supported features for large speech
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-large-speech-languages#models-lsm-supported-features)
* [Next-generation languages and
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Supported features for next-generation
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng#models-ng-features)
### Multipart speech recognition
**Note:** The asynchronous HTTP interface, WebSocket interface, and Watson SDKs
do not support multipart speech recognition.
The HTTP `POST` method of the service also supports multipart speech recognition.
With multipart requests, you pass all audio data as multipart form data. You
specify some parameters as request headers and query parameters, but you pass JSON
metadata as form data to control most aspects of the transcription. You can use
multipart recognition to pass multiple audio files with a single request.
Use the multipart approach with browsers for which JavaScript is disabled or when
the parameters used with the request are greater than the 8 KB limit imposed by
most HTTP servers and proxies. You can encounter this limit, for example, if you
want to spot a very large number of keywords.
**See also:** [Making a multipart HTTP
request](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-http#HTTP-multi).
:param BinaryIO audio: The audio to transcribe.
:param str content_type: (optional) The format (MIME type) of the audio.
For more information about specifying an audio format, see **Audio formats
(content types)** in the method description.
:param str model: (optional) The model to use for speech recognition. If
you omit the `model` parameter, the service uses the US English
`en-US_BroadbandModel` by default.
_For IBM Cloud Pak for Data,_ if you do not install the
`en-US_BroadbandModel`, you must either specify a model with the request or
specify a new default model for your installation of the service.
**See also:**
* [Using a model for speech
recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-use)
* [Using the default
model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-use#models-use-default).
:param bool speech_begin_event: (optional) If `true`, the service returns a
response object `SpeechActivity` which contains the time when a speech
activity is detected in the stream. This can be used both in standard and
low latency mode. This feature enables client applications to know that
some words/speech has been detected and the service is in the process of
decoding. This can be used in lieu of interim results in standard mode. Use
`sad_module: 2` to increase accuracy and performance in detecting speech
boundaries within the audio stream. See [Using speech recognition
parameters](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-service-features#features-parameters).
:param str language_customization_id: (optional) The customization ID
(GUID) of a custom language model that is to be used with the recognition
request. The base model of the specified custom language model must match
the model specified with the `model` parameter. You must make the request
with credentials for the instance of the service that owns the custom
model. By default, no custom language model is used. See [Using a custom
language model for speech
recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageUse).
**Note:** Use this parameter instead of the deprecated `customization_id`
parameter.
:param str acoustic_customization_id: (optional) The customization ID
(GUID) of a custom acoustic model that is to be used with the recognition
request. The base model of the specified custom acoustic model must match
the model specified with the `model` parameter. You must make the request
with credentials for the instance of the service that owns the custom
model. By default, no custom acoustic model is used. See [Using a custom
acoustic model for speech
recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-acousticUse).
:param str base_model_version: (optional) The version of the specified base
model that is to be used with the recognition request. Multiple versions of
a base model can exist when a model is updated for internal improvements.
The parameter is intended primarily for use with custom models that have
been upgraded for a new base model. The default value depends on whether
the parameter is used with or without a custom model. See [Making speech
recognition requests with upgraded custom
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-custom-upgrade-use#custom-upgrade-use-recognition).
:param float customization_weight: (optional) If you specify the
customization ID (GUID) of a custom language model with the recognition
request, the customization weight tells the service how much weight to give
to words from the custom language model compared to those from the base
model for the current request.
Specify a value between 0.0 and 1.0. Unless a different customization
weight was specified for the custom model when the model was trained, the
default value is:
* 0.5 for large speech models
* 0.3 for previous-generation models
* 0.2 for most next-generation models
* 0.1 for next-generation English and Japanese models
A customization weight that you specify overrides a weight that was
specified when the custom model was trained. The default value yields the
best performance in general. Assign a higher value if your audio makes
frequent use of OOV words from the custom model. Use caution when setting
the weight: a higher value can improve the accuracy of phrases from the
custom model's domain, but it can negatively affect performance on
non-domain phrases.
See [Using customization
weight](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-languageUse#weight).
:param int inactivity_timeout: (optional) The time in seconds after which,
if only silence (no speech) is detected in streaming audio, the connection
is closed with a 400 error. The parameter is useful for stopping audio
submission from a live microphone when a user simply walks away. Use `-1`
for infinity. See [Inactivity
timeout](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-input#timeouts-inactivity).
:param List[str] keywords: (optional) An array of keyword strings to spot
in the audio. Each keyword string can include one or more string tokens.
Keywords are spotted only in the final results, not in interim hypotheses.
If you specify any keywords, you must also specify a keywords threshold.
Omit the parameter or specify an empty array if you do not need to spot
keywords.
You can spot a maximum of 1000 keywords with a single request. A single
keyword can have a maximum length of 1024 characters, though the maximum
effective length for double-byte languages might be shorter. Keywords are
case-insensitive.
See [Keyword
spotting](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-spotting#keyword-spotting).
:param float keywords_threshold: (optional) A confidence value that is the
lower bound for spotting a keyword. A word is considered to match a keyword
if its confidence is greater than or equal to the threshold. Specify a
probability between 0.0 and 1.0. If you specify a threshold, you must also
specify one or more keywords. The service performs no keyword spotting if
you omit either parameter. See [Keyword
spotting](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-spotting#keyword-spotting).
:param int max_alternatives: (optional) The maximum number of alternative
transcripts that the service is to return. By default, the service returns
a single transcript. If you specify a value of `0`, the service uses the
default value, `1`. See [Maximum
alternatives](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metadata#max-alternatives).
:param float word_alternatives_threshold: (optional) A confidence value
that is the lower bound for identifying a hypothesis as a possible word
alternative (also known as "Confusion Networks"). An alternative word is
considered if its confidence is greater than or equal to the threshold.
Specify a probability between 0.0 and 1.0. By default, the service computes
no alternative words. See [Word
alternatives](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-spotting#word-alternatives).
:param bool word_confidence: (optional) If `true`, the service returns a
confidence measure in the range of 0.0 to 1.0 for each word. By default,
the service returns no word confidence scores. See [Word
confidence](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metadata#word-confidence).
:param bool timestamps: (optional) If `true`, the service returns time
alignment for each word. By default, no timestamps are returned. See [Word
timestamps](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metadata#word-timestamps).
:param bool profanity_filter: (optional) If `true`, the service filters
profanity from all output except for keyword results by replacing
inappropriate words with a series of asterisks. Set the parameter to
`false` to return results with no censoring.
**Note:** The parameter can be used with US English and Japanese
transcription only. See [Profanity
filtering](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formatting#profanity-filtering).
:param bool smart_formatting: (optional) If `true`, the service converts
dates, times, series of digits and numbers, phone numbers, currency values,
and internet addresses into more readable, conventional representations in
the final transcript of a recognition request. For US English, the service
also converts certain keyword strings to punctuation symbols. By default,
the service performs no smart formatting.
**Note:** The parameter can be used with US English, Japanese, and Spanish
(all dialects) transcription only.
See [Smart
formatting](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formatting#smart-formatting).
:param int smart_formatting_version: (optional) Smart formatting version
for large speech models and next-generation models is supported in US
English, Brazilian Portuguese, French, German, Spanish and French Canadian
languages.
:param bool speaker_labels: (optional) If `true`, the response includes
labels that identify which words were spoken by which participants in a
multi-person exchange. By default, the service returns no speaker labels.
Setting `speaker_labels` to `true` forces the `timestamps` parameter to be
`true`, regardless of whether you specify `false` for the parameter.
* _For previous-generation models,_ the parameter can be used with
Australian English, US English, German, Japanese, Korean, and Spanish (both
broadband and narrowband models) and UK English (narrowband model)
transcription only.
* _For large speech models and next-generation models,_ the parameter can
be used with all available languages.
See [Speaker
labels](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-speaker-labels).
:param str grammar_name: (optional) The name of a grammar that is to be
used with the recognition request. If you specify a grammar, you must also
use the `language_customization_id` parameter to specify the name of the
custom language model for which the grammar is defined. The service
recognizes only strings that are recognized by the specified grammar; it
does not recognize other custom words from the model's words resource.
See [Using a grammar for speech
recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-grammarUse).
:param bool redaction: (optional) If `true`, the service redacts, or masks,
numeric data from final transcripts. The feature redacts any number that
has three or more consecutive digits by replacing each digit with an `X`
character. It is intended to redact sensitive numeric data, such as credit
card numbers. By default, the service performs no redaction.
When you enable redaction, the service automatically enables smart
formatting, regardless of whether you explicitly disable that feature. To
ensure maximum security, the service also disables keyword spotting
(ignores the `keywords` and `keywords_threshold` parameters) and returns
only a single final transcript (forces the `max_alternatives` parameter to
be `1`).
**Note:** The parameter can be used with US English, Japanese, and Korean
transcription only.
See [Numeric
redaction](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-formatting#numeric-redaction).
:param bool audio_metrics: (optional) If `true`, requests detailed
information about the signal characteristics of the input audio. The
service returns audio metrics with the final transcription results. By
default, the service returns no audio metrics.
See [Audio
metrics](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-metrics#audio-metrics).
:param float end_of_phrase_silence_time: (optional) Specifies the duration
of the pause interval at which the service splits a transcript into
multiple final results. If the service detects pauses or extended silence
before it reaches the end of the audio stream, its response can include
multiple final results. Silence indicates a point at which the speaker
pauses between spoken words or phrases.
Specify a value for the pause interval in the range of 0.0 to 120.0.
* A value greater than 0 specifies the interval that the service is to use
for speech recognition.
* A value of 0 indicates that the service is to use the default interval.
It is equivalent to omitting the parameter.
The default pause interval for most languages is 0.8 seconds; the default
for Chinese is 0.6 seconds.
See [End of phrase silence
time](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsing#silence-time).
:param bool split_transcript_at_phrase_end: (optional) If `true`, directs
the service to split the transcript into multiple final results based on
semantic features of the input, for example, at the conclusion of
meaningful phrases such as sentences. The service bases its understanding
of semantic features on the base language model that you use with a
request. Custom language models and grammars can also influence how and
where the service splits a transcript.
By default, the service splits transcripts based solely on the pause
interval. If the parameters are used together on the same request,
`end_of_phrase_silence_time` has precedence over
`split_transcript_at_phrase_end`.
See [Split transcript at phrase
end](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsing#split-transcript).
:param float speech_detector_sensitivity: (optional) The sensitivity of
speech activity detection that the service is to perform. Use the parameter
to suppress word insertions from music, coughing, and other non-speech
events. The service biases the audio it passes for speech recognition by
evaluating the input audio against prior models of speech and non-speech
activity.
Specify a value between 0.0 and 1.0:
* 0.0 suppresses all audio (no speech is transcribed).
* 0.5 (the default) provides a reasonable compromise for the level of
sensitivity.
* 1.0 suppresses no audio (speech detection sensitivity is disabled).
The values increase on a monotonic curve. Specifying one or two decimal
places of precision (for example, `0.55`) is typically more than
sufficient.
The parameter is supported with all large speech models, next-generation
models and with most previous-generation models. See [Speech detector
sensitivity](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detection#detection-parameters-sensitivity)
and [Language model
support](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detection#detection-support).
:param int sad_module: (optional) Detects speech boundaries within the
audio stream with better performance, improved noise suppression, faster
responsiveness, and increased accuracy.
Specify `sad_module: 2`
See [Speech Activity Detection
(SAD)](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detection#sad).
:param float background_audio_suppression: (optional) The level to which
the service is to suppress background audio based on its volume to prevent
it from being transcribed as speech. Use the parameter to suppress side
conversations or background noise.
Specify a value in the range of 0.0 to 1.0:
* 0.0 (the default) provides no suppression (background audio suppression
is disabled).
* 0.5 provides a reasonable level of audio suppression for general usage.
* 1.0 suppresses all audio (no audio is transcribed).
The values increase on a monotonic curve. Specifying one or two decimal
places of precision (for example, `0.55`) is typically more than
sufficient.
The parameter is supported with all large speech models, next-generation
models and with most previous-generation models. See [Background audio
suppression](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detection#detection-parameters-suppression)
and [Language model
support](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-detection#detection-support).
:param bool low_latency: (optional) If `true` for next-generation
`Multimedia` and `Telephony` models that support low latency, directs the
service to produce results even more quickly than it usually does.
Next-generation models produce transcription results faster than
previous-generation models. The `low_latency` parameter causes the models
to produce results even more quickly, though the results might be less
accurate when the parameter is used.
The parameter is not available for large speech models and
previous-generation `Broadband` and `Narrowband` models. It is available
for most next-generation models.
* For a list of next-generation models that support low latency, see
[Supported next-generation language
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng#models-ng-supported).
* For more information about the `low_latency` parameter, see [Low
latency](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-interim#low-latency).
:param float character_insertion_bias: (optional) For large speech models
and next-generation models, an indication of whether the service is biased
to recognize shorter or longer strings of characters when developing
transcription hypotheses. By default, the service is optimized to produce
the best balance of strings of different lengths.
The default bias is 0.0. The allowable range of values is -1.0 to 1.0.
* Negative values bias the service to favor hypotheses with shorter strings
of characters.
* Positive values bias the service to favor hypotheses with longer strings
of characters.
As the value approaches -1.0 or 1.0, the impact of the parameter becomes
more pronounced. To determine the most effective value for your scenario,
start by setting the value of the parameter to a small increment, such as
-0.1, -0.05, 0.05, or 0.1, and assess how the value impacts the
transcription results. Then experiment with different values as necessary,
adjusting the value by small increments.
The parameter is not available for previous-generation models.
See [Character insertion
bias](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-parsing#insertion-bias).
:param dict headers: A `dict` containing the request headers
:return: A `DetailedResponse` containing the result, headers and HTTP status code.
:rtype: DetailedResponse with `dict` result representing a `SpeechRecognitionResults` object
"""
if audio is None:
raise ValueError('audio must be provided')
headers = {
'Content-Type': content_type,
}
sdk_headers = get_sdk_headers(
service_name=self.DEFAULT_SERVICE_NAME,
service_version='V1',
operation_id='recognize',
)
headers.update(sdk_headers)
params = {
'model': model,
'speech_begin_event': speech_begin_event,
'language_customization_id': language_customization_id,
'acoustic_customization_id': acoustic_customization_id,
'base_model_version': base_model_version,
'customization_weight': customization_weight,
'inactivity_timeout': inactivity_timeout,
'keywords': convert_list(keywords),
'keywords_threshold': keywords_threshold,
'max_alternatives': max_alternatives,
'word_alternatives_threshold': word_alternatives_threshold,
'word_confidence': word_confidence,
'timestamps': timestamps,
'profanity_filter': profanity_filter,
'smart_formatting': smart_formatting,
'smart_formatting_version': smart_formatting_version,
'speaker_labels': speaker_labels,
'grammar_name': grammar_name,
'redaction': redaction,
'audio_metrics': audio_metrics,
'end_of_phrase_silence_time': end_of_phrase_silence_time,
'split_transcript_at_phrase_end': split_transcript_at_phrase_end,
'speech_detector_sensitivity': speech_detector_sensitivity,
'sad_module': sad_module,
'background_audio_suppression': background_audio_suppression,
'low_latency': low_latency,
'character_insertion_bias': character_insertion_bias,
}
data = audio
if 'headers' in kwargs:
headers.update(kwargs.get('headers'))
del kwargs['headers']
headers['Accept'] = 'application/json'
url = '/v1/recognize'
request = self.prepare_request(
method='POST',
url=url,
headers=headers,
params=params,
data=data,
)
response = self.send(request, **kwargs)
return response
#########################
# Asynchronous
#########################
def register_callback(
self,
callback_url: str,
*,
user_secret: Optional[str] = None,
**kwargs,
) -> DetailedResponse:
"""
Register a callback.
Registers a callback URL with the service for use with subsequent asynchronous
recognition requests. The service attempts to register, or allowlist, the callback
URL if it is not already registered by sending a `GET` request to the callback
URL. The service passes a random alphanumeric challenge string via the
`challenge_string` parameter of the request. The request includes an `Accept`
header that specifies `text/plain` as the required response type.
To be registered successfully, the callback URL must respond to the `GET` request
from the service. The response must send status code 200 and must include the
challenge string in its body. Set the `Content-Type` response header to
`text/plain`. Upon receiving this response, the service responds to the original
registration request with response code 201.
The service sends only a single `GET` request to the callback URL. If the service
does not receive a reply with a response code of 200 and a body that echoes the
challenge string sent by the service within five seconds, it does not allowlist
the URL; it instead sends status code 400 in response to the request to register a
callback. If the requested callback URL is already allowlisted, the service
responds to the initial registration request with response code 200.
If you specify a user secret with the request, the service uses it as a key to
calculate an HMAC-SHA1 signature of the challenge string in its response to the
`POST` request. It sends this signature in the `X-Callback-Signature` header of
its `GET` request to the URL during registration. It also uses the secret to
calculate a signature over the payload of every callback notification that uses
the URL. The signature provides authentication and data integrity for HTTP
communications.
After you successfully register a callback URL, you can use it with an indefinite
number of recognition requests. You can register a maximum of 20 callback URLS in
a one-hour span of time.
**See also:** [Registering a callback
URL](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-async#register).
:param str callback_url: An HTTP or HTTPS URL to which callback
notifications are to be sent. To be allowlisted, the URL must successfully
echo the challenge string during URL verification. During verification, the
client can also check the signature that the service sends in the
`X-Callback-Signature` header to verify the origin of the request.
:param str user_secret: (optional) A user-specified string that the service
uses to generate the HMAC-SHA1 signature that it sends via the
`X-Callback-Signature` header. The service includes the header during URL
verification and with every notification sent to the callback URL. It
calculates the signature over the payload of the notification. If you omit
the parameter, the service does not send the header.
:param dict headers: A `dict` containing the request headers
:return: A `DetailedResponse` containing the result, headers and HTTP status code.
:rtype: DetailedResponse with `dict` result representing a `RegisterStatus` object
"""
if not callback_url:
raise ValueError('callback_url must be provided')
headers = {}
sdk_headers = get_sdk_headers(
service_name=self.DEFAULT_SERVICE_NAME,
service_version='V1',
operation_id='register_callback',
)
headers.update(sdk_headers)
params = {
'callback_url': callback_url,
'user_secret': user_secret,
}
if 'headers' in kwargs:
headers.update(kwargs.get('headers'))
del kwargs['headers']
headers['Accept'] = 'application/json'
url = '/v1/register_callback'
request = self.prepare_request(
method='POST',
url=url,
headers=headers,
params=params,
)
response = self.send(request, **kwargs)
return response
def unregister_callback(
self,
callback_url: str,
**kwargs,
) -> DetailedResponse:
"""
Unregister a callback.
Unregisters a callback URL that was previously allowlisted with a [Register a
callback](#registercallback) request for use with the asynchronous interface. Once
unregistered, the URL can no longer be used with asynchronous recognition
requests.
**See also:** [Unregistering a callback
URL](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-async#unregister).
:param str callback_url: The callback URL that is to be unregistered.
:param dict headers: A `dict` containing the request headers
:return: A `DetailedResponse` containing the result, headers and HTTP status code.
:rtype: DetailedResponse
"""
if not callback_url:
raise ValueError('callback_url must be provided')
headers = {}
sdk_headers = get_sdk_headers(
service_name=self.DEFAULT_SERVICE_NAME,
service_version='V1',
operation_id='unregister_callback',
)
headers.update(sdk_headers)
params = {
'callback_url': callback_url,
}
if 'headers' in kwargs:
headers.update(kwargs.get('headers'))
del kwargs['headers']
url = '/v1/unregister_callback'
request = self.prepare_request(
method='POST',
url=url,
headers=headers,
params=params,
)
response = self.send(request, **kwargs)
return response
def create_job(
self,
audio: BinaryIO,
*,
content_type: Optional[str] = None,
model: Optional[str] = None,
callback_url: Optional[str] = None,
events: Optional[str] = None,
user_token: Optional[str] = None,
results_ttl: Optional[int] = None,
language_customization_id: Optional[str] = None,
acoustic_customization_id: Optional[str] = None,
base_model_version: Optional[str] = None,
customization_weight: Optional[float] = None,
inactivity_timeout: Optional[int] = None,
keywords: Optional[List[str]] = None,
keywords_threshold: Optional[float] = None,
max_alternatives: Optional[int] = None,
word_alternatives_threshold: Optional[float] = None,
word_confidence: Optional[bool] = None,
timestamps: Optional[bool] = None,
profanity_filter: Optional[bool] = None,
smart_formatting: Optional[bool] = None,
smart_formatting_version: Optional[int] = None,
speaker_labels: Optional[bool] = None,
grammar_name: Optional[str] = None,
redaction: Optional[bool] = None,
processing_metrics: Optional[bool] = None,
processing_metrics_interval: Optional[float] = None,
audio_metrics: Optional[bool] = None,
end_of_phrase_silence_time: Optional[float] = None,
split_transcript_at_phrase_end: Optional[bool] = None,
speech_detector_sensitivity: Optional[float] = None,
sad_module: Optional[int] = None,
background_audio_suppression: Optional[float] = None,
low_latency: Optional[bool] = None,
character_insertion_bias: Optional[float] = None,
**kwargs,
) -> DetailedResponse:
"""
Create a job.
Creates a job for a new asynchronous recognition request. The job is owned by the
instance of the service whose credentials are used to create it. How you learn the
status and results of a job depends on the parameters you include with the job
creation request:
* By callback notification: Include the `callback_url` parameter to specify a URL
to which the service is to send callback notifications when the status of the job
changes. Optionally, you can also include the `events` and `user_token` parameters
to subscribe to specific events and to specify a string that is to be included
with each notification for the job.
* By polling the service: Omit the `callback_url`, `events`, and `user_token`
parameters. You must then use the [Check jobs](#checkjobs) or [Check a
job](#checkjob) methods to check the status of the job, using the latter to
retrieve the results when the job is complete.
The two approaches are not mutually exclusive. You can poll the service for job
status or obtain results from the service manually even if you include a callback
URL. In both cases, you can include the `results_ttl` parameter to specify how
long the results are to remain available after the job is complete. Using the
HTTPS [Check a job](#checkjob) method to retrieve results is more secure than
receiving them via callback notification over HTTP because it provides
confidentiality in addition to authentication and data integrity.
The method supports the same basic parameters as other HTTP and WebSocket
recognition requests. It also supports the following parameters specific to the
asynchronous interface:
* `callback_url`
* `events`
* `user_token`
* `results_ttl`
You can pass a maximum of 1 GB and a minimum of 100 bytes of audio with a request.
The service automatically detects the endianness of the incoming audio and, for
audio that includes multiple channels, downmixes the audio to one-channel mono
during transcoding. The method returns only final results; to enable interim
results, use the WebSocket API. (With the `curl` command, use the `--data-binary`
option to upload the file for the request.)
**See also:** [Creating a
job](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-async#create).
### Streaming mode
For requests to transcribe live audio as it becomes available, you must set the
`Transfer-Encoding` header to `chunked` to use streaming mode. In streaming mode,
the service closes the connection (status code 408) if it does not receive at
least 15 seconds of audio (including silence) in any 30-second period. The service
also closes the connection (status code 400) if it detects no speech for
`inactivity_timeout` seconds of streaming audio; use the `inactivity_timeout`
parameter to change the default of 30 seconds.
**See also:**
* [Audio
transmission](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-input#transmission)
*
[Timeouts](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-input#timeouts)
### Audio formats (content types)
The service accepts audio in the following formats (MIME types).
* For formats that are labeled **Required**, you must use the `Content-Type`
header with the request to specify the format of the audio.
* For all other formats, you can omit the `Content-Type` header or specify
`application/octet-stream` with the header to have the service automatically
detect the format of the audio. (With the `curl` command, you can specify either
`"Content-Type:"` or `"Content-Type: application/octet-stream"`.)
Where indicated, the format that you specify must include the sampling rate and
can optionally include the number of channels and the endianness of the audio.
* `audio/alaw` (**Required.** Specify the sampling rate (`rate`) of the audio.)
* `audio/basic` (**Required.** Use only with narrowband models.)
* `audio/flac`
* `audio/g729` (Use only with narrowband models.)
* `audio/l16` (**Required.** Specify the sampling rate (`rate`) and optionally the
number of channels (`channels`) and endianness (`endianness`) of the audio.)
* `audio/mp3`
* `audio/mpeg`
* `audio/mulaw` (**Required.** Specify the sampling rate (`rate`) of the audio.)
* `audio/ogg` (The service automatically detects the codec of the input audio.)
* `audio/ogg;codecs=opus`
* `audio/ogg;codecs=vorbis`
* `audio/wav` (Provide audio with a maximum of nine channels.)
* `audio/webm` (The service automatically detects the codec of the input audio.)
* `audio/webm;codecs=opus`
* `audio/webm;codecs=vorbis`
The sampling rate of the audio must match the sampling rate of the model for the
recognition request: for broadband models, at least 16 kHz; for narrowband models,
at least 8 kHz. If the sampling rate of the audio is higher than the minimum
required rate, the service down-samples the audio to the appropriate rate. If the
sampling rate of the audio is lower than the minimum required rate, the request
fails.
**See also:** [Supported audio
formats](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-audio-formats).
### Large speech models and Next-generation models
The service supports large speech models and next-generation `Multimedia` (16
kHz) and `Telephony` (8 kHz) models for many languages. Large speech models and
next-generation models have higher throughput than the service's previous
generation of `Broadband` and `Narrowband` models. When you use large speech
models and next-generation models, the service can return transcriptions more
quickly and also provide noticeably better transcription accuracy.
You specify a large speech model or next-generation model by using the `model`
query parameter, as you do a previous-generation model. Only the next-generation
models support the `low_latency` parameter, and all large speech models and
next-generation models support the `character_insertion_bias` parameter. These
parameters are not available with previous-generation models.
Large speech models and next-generation models do not support all of the speech
recognition parameters that are available for use with previous-generation models.
Next-generation models do not support the following parameters:
* `acoustic_customization_id`
* `keywords` and `keywords_threshold`
* `processing_metrics` and `processing_metrics_interval`
* `word_alternatives_threshold`
**Important:** Effective **31 July 2023**, all previous-generation models will be
removed from the service and the documentation. Most previous-generation models
were deprecated on 15 March 2022. You must migrate to the equivalent large speech
model or next-generation model by 31 July 2023. For more information, see
[Migrating to large speech
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate).
**See also:**
* [Large speech languages and
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-large-speech-languages)
* [Supported features for large speech
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-large-speech-languages#models-lsm-supported-features)
* [Next-generation languages and
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng)
* [Supported features for next-generation
models](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng#models-ng-features).
:param BinaryIO audio: The audio to transcribe.
:param str content_type: (optional) The format (MIME type) of the audio.
For more information about specifying an audio format, see **Audio formats
(content types)** in the method description.
:param str model: (optional) The model to use for speech recognition. If
you omit the `model` parameter, the service uses the US English
`en-US_BroadbandModel` by default.
_For IBM Cloud Pak for Data,_ if you do not install the
`en-US_BroadbandModel`, you must either specify a model with the request or
specify a new default model for your installation of the service.
**See also:**
* [Using a model for speech
recognition](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-use)
* [Using the default
model](https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-use#models-use-default).
:param str callback_url: (optional) A URL to which callback notifications
are to be sent. The URL must already be successfully allowlisted by using
the [Register a callback](#registercallback) method. You can include the